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[Corresponding to PCT/JP99/05736 
Filed October 19, 1999] 



Attn: BOX PCX 

Docket No. 2000 0759A 



FIRST PRELIMINARY AMENDMENT 



Assistant Commissioner for Patents, 
Washington, DC 20231 



Please amend the above-identified application as follows. 



In the Claims : 

Kindly amend claims 3, 4 and 10-12 as follows. 
Claim 3, line 1, delete "or 2"; 
Claim 4, line 1, delete "or 2"; 
Claim 10, line 1, delete "or 9"; 
Claim 1 1, line 1, delete "or 9"; 
Claim 12, line 1, delete "or S 



Kindly add new claims 41-45 as follows. 

41 . The network control system according to claim 2, wherein 

said identification information includes version information indicating a version of 
said screen display data. 



ATTACHMENT F 



42. The network control system according to claim 2, wherein 

said operation information includes an operating position information indicative of 
an operating position on said operating screen. 

43 . The network control system according to claim 9, wherein 

said partial screen identification information includes version information indicating 
a version of said partial screen display data. 

44. The network control system according to claim 9, wherein 

one display element in screen display is arranged in any one of a plurality of display 
parts corresponding to said plurality of partial screen display data. 

45. The network control system according to claim 9, wherein 

a display element corresponding to each operation by the user is arranged in any 
one of a plurality of display parts corresponding to said plurality of partial screen display data. 

REMARKS 

The above claim amendments are presented in order to remove multiple claim dependency, 
so as to reduce the required filing fee. 

RespectfixUy submitted, 
Yoshifiimi YANAGAWA et al. 



By ^C^^>h6J^ 
Charles R. Watts 
Registration No . 3 3 , 1 42 
Attorney for Applicants 

CRW/asd 
Washington, D.C. 
Telephone (202) 721-8200 
Facsimile (202) 721-8250 
June 16, 2000 
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[Corresponding to PCT/JP99/05736 
Filed October 19, 1999] 



SECOND PRELIMINARY AMENDMENT 



Assistant Commissioner for Patents, 
Washington, DC 20231 



Sir: 

Please amend the above-identified application as follows. 



In the Specification and Abstract : 

Page 62, line 4, before "generator 91" insert -information-; 
Page 65, line 5, after "provided or not" insert -(step S705)~; 
Page 72, line 16, change "241b" to -241c~; 

line 17, change "241b" to -241c-; 

line 20, change "241b" to ~241c~; 
Page 75, line 17, after "element" insert -information 80—. 



ATTACHMENT G 



REMARKS 

Please enter the above amendments prior to initial examination. 

Respectfully submitted, 

Yoshifumi YANAGAWA et al. 



By ^i>^ ^ CiytiF>^^^ 
Charles R. Watts 
Registration No . 3 3 , 1 42 
Attorney for Applicants 

CRW/asd 
Washington, D.C. 
Telephone (202) 721-8200 
Facsimile (202) 721-8250 
June 16, 2000 
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SPECIFICATION 



N ETWORK CONTROL SYSTEM 



TECHNICAL FIELD 



5 



The present invention relates to network control systems 



in which a device connected to a network is operated through a 
network and, more specifically, a device control system with a 
graphical user interface ( GUI ) supporting user ' s device operation 
through graphics, characters, etc., on a screen. 



In recent years, device control systems for controlling 
devices by selectively operating, through a remote controller, 
device control graphics (icons) composed of screen display data 

15 and characters indicating functions of the devices on a TV 
(Television) screen have made their appearance. Also, network 
systems with digital devices such as a DVD connected thereto using 
IEEE 1394-1995 for transmitting and receiving video/audio data 
have been emerging. 

20 One example of conventional network control systems is 

disclosed in Japanese Patent Laying-Open No. H9- 149325 (1997- 
149325) . 

This conventional network control system is now described 

below. 

25 Similarly to digital interfaces according to IEEE 1394 



10 



BACKGROUND ART 



1 



ATTACHMENT B 



standards , etc . , AV units are connected to each other without 
switching by a bilateral packet communications system with a 
serial bus that periodically provides an equal communication 
opportunity to the devices . 
5 Here, each AV unit stores its own screen display data. By 

request from a controller having a graphic display function 
(television receiver) , each AV unit transmits the screen display 
data to the controller. The controller displays the received 
screen display data. The controller further has the functions 
10 of inquiring for data required for indicating the AV unit 
connected thereto and controlling the display screen based on the 
screen display data from the AV unit . Each AV unit has a recording 
medium for storing the screen display data, and a function of 
selecting the appropriate screen display data in response to the 
15 inquiry from the controller for the screen display data. 

In the above structured network control system, the screen 
display data is held in each device (AV unit), and outputted 
according to a display request from the controller (television 
receiver). Graphics unique to each device (AV unit) are thus 
20 displayed on the screen of the controller. 

In the above described structure, however, when the display 
on an operating screen is to be changed due to internal state 
changes in the device, the device cannot report its change of state 
to the controller, thereby causing a mismatch in display screen 
25 information between the controller and the device. For this 



reason, the controller cannot correctly provide a user with device 
operation information. 

In view of the above problem, an object of the present 
invention is to provide a network control system in which a device 

5 can quickly report to a controller about a change of the device ' s 
internal state that causes a change of display on an operating 
screen to ensure the controller and the device can share the scime 
state information. Such network control system is herein 
achieved with simple structure and small transmission load on a 

10 communication path. 

DISCLOSURE OF THE INVENTION 
A first aspect of the present invention is directed to a 
network control system in which a first unit and a second unit 
15 are connected to each other a transmission path and a controller 
included in the first unit controls a device in the second unit 
through the transmission path, 

at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, 
20 the device 

having screen display data for displaying an operating 
screen of the device and identification information for 
identifying the screen display data; and 

transmitting the screen display data and the 
25 identification information to the controller through the 



transmission path, and 
the controller 

comprising a user interface including display means; 
receiving the screen display data and the identification 
5 information from the device through the transmission path; 

displaying the operating screen on the display means using 
the screen display data; and 

in response to an operation by a user to the operating 
screen, controlling the device by transmitting operation 
10 information indicative of the operation and the identification 
information to the device through the transmission path. 

A second aspect of the present invention is directed to a 
network control system in which a first unit and a second unit 
are connected to each other a transmission path and a controller 
15 included in the first unit controls a device in the second unit 
through the transmission path, 

at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, 
the device 

20 having screen display data for displaying an operating 

screen of the device and identification information for 
identifying the screen display data; and 

transmitting the screen display data and the 
identification information to the controller through the 

25 transmission path; and 



the controller 

comprising a user interface including display means; 
receiving the screen display data and the identification 
information from the device through the transmission path; 
5 displaying the operating screen on the display means using 

the screen display data and the identification information; and 
in response to an operation by a user to the operating 
screen, controlling the device by transmitting operation 
information indicative of the operation and the identification 
10 information to the device through the transmission path. 

According to the first or second aspect of the present 
invention, by identifying the screen display data with the 
identification information, the device can easily and reliably 
recognize to which screen display data the user operation 
15 information sent by the controller is directed, thereby 
preventing erroneous operation of the device. Further, 
according to the first or second aspect, the controller manages 
the screen display data representing the operating screen of the 
device, and transmits the operation information based on the 
20 user's operation. However, the controller does not have to 
understand the meaning of the function of the device or the 
operation (the relation between the operation and the function 
of the device). In other words, user interface information 
(screen display data, operation information, etc.,) is managed 
25 and processed by the controller, while control information 



directly related to device control (information indicating the 
function of the device, information indicating the relation 
between the function of the device and the operation, etc. ,) is 
by the device. Therefore, the processing in the controller is 

5 reduced, and the types of operation for controlling the device 
by the controller are not limited but can be varied. Furthermore, 
according to the first or second aspect, the controller obtains 
the screen display data representing the operating screen of the 
device and the identification information from the device and 

10 display the operating screen of the device using the obtained data 
and information. Therefore, even if the device has a new function 
that cannot be presently assumed, only by sending the screen 
display data corresponding the new function from the device to 
the controller, the new function can be easily provided for and 

15 is made available to the user. 

According to a third aspect of the present invention, in 
the first or second aspect, 

the identification information includes version 
information indicating a version of the screen display data. 

20 According to the third aspect of the present invention, the 

identification information obtained by the controller from the 
device includes the version information. Therefore, the 
controller can detect a change in the operating screen information 
of the device, and thereby can easily detects that the state of 

25 the device is changed based on instructions from other controller 



or voluntary change in the device. Further, according to the 
user's operation, the version information of the screen display 
data representing the operating screen is sent with the operation 
information to the device. Therefore, the device can easily and 

5 reliably understand to which operating screen the operation is 
directed, thereby executing the operation desired by the user. 
Further, according to the third aspect, when receiving the screen 
display data, the controller checks the version information 
thereof, and updates display only when the version information 

10 is correctly updated. Therefore, it is possible to reduce a load 
on the controller when the same screen display data is transmitted 
again and again. 

According to a fourth aspect of the present invention, in 
the first or second aspect, 

15 the operation information includes an operating position 

information indicative of an operating position on the operating 
screen . 

According to the fourth aspect of the present invention, 
the operation information that is sent to the device according 

20 to the user's operation includes the operating position 
information indicating the operating position on the operating 
screen. Therefore, in the operation to the operating screen, a 
desired operating position can be directly specified by a pointing 
device. Thus, operability for device control is improved. 

25 A fifth aspect of the present invention is directed to a 



network control system in which a first unit and a second unit 
are connected to each other a transmission path and a controller 
included in the first unit controls a device in the second unit 
through the transmission path, 
5 at least one of the first unit and the second unit being 

for handling at least one of video, audio, and information, 
the device 

having screen display data composed of a plurality of 
partial screen display data for displaying an operating screen 
10 of the device; and 

transmitting the partial screen display data to the 
controller through the transmission path, and 
the controller 

comprising a user interface including display means; 
15 receiving the partial screen display data from the device 

through the transmission path; 

displaying the operating screen on the display means using 
the partial screen display data; and 

in response to an operation by a user to the operating 
20 screen, controlling the device by transmitting operation 
information indicative of the operation to the device through the 
transmission path. 

According to the fifth aspect of the present invention, the 
screen display data can be constructed of a plurality of partial 
25 screen display data obtained by dividing the screen, thereby 



reducing the data amount to be transferred. 

According to a sixth aspect of the present invention, in 
the fifth aspect, 

when the screen display data of the device is changed, the 
5 device transmits changed partial screen display data of the screen 
display data to the controller; and 

the controller receives the changed partial screen display 
data from the device through the transmission path and, based on 
the received partial screen display data, updates the operating 
10 screen displayed on the display means . 

According to the sixth aspect of the present invention, when 
the screen display data of the device is changed because the state 
in the device is changed, for example, only the partial screen 
display data including the changed display element is sent to the 
15 controller. Therefore, the data amount to be transferred to the 
controller on change of screen display is reduced, and the 
processing in the device and controller is simplified. 

According to a seventh aspect of the present invention, in 
the fifth aspect, 
20 the device 

has partial screen identification information for 
identifying the partial screen display data; and 

transmits the partial screen display data and the partial 
screen identification information to the controller through the 
25 transmission path, and 



the controller 

receives the partial screen display data and the partial 
screen identification information from the device through the 
transmission path. 
5 According to an eighth aspect of the present invention, in 

the seventh aspect, 

when the screen display data of the device is changed, the 
device transmits changed partial screen display data of the screen 
display data and the partial screen identification information 
10 of the partial screen display data to the controller, and 

the controller receives the changed partial screen display 
data and the partial screen identification information of the 
partial screen display data from the device through the 
transmission path and, based on the received partial screen 
15 display data and partial screen identification information, 
updates the operating screen displayed on the display means. 

According to the eighth aspect of the present invention, 
when the screen display data of the device is changed, not only 
the partial screen display data including the change display 
20 element but also its partial screen identification information 
is sent to the controller. Therefore, the controller can easily 
specify the changed partial screen display data, thereby reducing 
a processing load such as updating the display of the operating 
screen on change of the screen display data. 
25 According to a ninth aspect of the present invention, in 



the seventh aspect , 

in response to an operation by a user to the operating screen, 
the controller controls the device by transmitting operation 
information indicative of the operation and the partial screen 
5 identification information corresponding to the operation to the 
device through the transmission path. 

According to the ninth aspect of the present invention, in 
response to an operation from the user, the operation information 
and the partial screen identification information corresponding 
10 to the operation are transmitted to the device. Therefore the 
device can easily and reliably recognize to which partial screen 
display data the operation information transmitted to the device 
is directed, thereby preventing erroneous operation of the 
device . 

15 According to a tenth aspect of the present invention, in 

the seventh or ninth aspect, 

the partial screen identification information includes 
version information indicating a version of the partial screen 
display data. 

20 According to the tenth aspect of the present invention, the 

partial screen identification information, which is sent to the 
controller together with the partial screen display data when the 
screen display data of the device is changed, includes version 
information. Therefore, the controller can prevent erroneous 

25 display of the operating screen by using this version information . 



Furthermore, in response to an operation from the user, the 
version information of the partial screen display data 
corresponding to the operation is sent to the device together with 
the operation information. Therefore, the device can easily and 
5 reliably know the operation by the user, thereby executing the 
operation as the user desires. 

According to an eleventh aspect of the present invention, 
in the seventh or ninth aspect, 

one display element in screen display is arranged in any 

10 one of a plurality of display parts corresponding to the plurality 
of partial screen display data. 

According to the eleventh aspect of the present invention, 
if any one of the display elements on the operating screen is 
changed, only the partial screen display data of that display 

15 element is sent from the device to the controller, and the 
plurality of partial screen display data do not have to be sent. 
Therefore, the data amount to be transferred to the controller 
on change of screen display is reduced, and also the processing 
in the device and controller is simplified. 

20 According to a twelfth aspect of the present invention, in 

the seventh or ninth aspect , 

a display element corresponding to each operation by the 
user is arranged in any one of a plurality of display parts 
corresponding to the plurality of partial screen display data. 

25 According to the twelfth aspect of the present invention. 



the display element corresponding to each operation by the user 
Is arranged in any one of the plurality of display parts 
corresponding to the plurality of partial screen display data. 
Therefore, when, in response to the operation from the user, the 
5 operation information and the partial screen identification 
information corresponding to that operation are sent to the device, 
the device can easily recognize the user operation from the 
partial screen identification information. 

A thirteenth aspect of the present invention is directed 
10 to a network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in the first unit controls a device in the 
second unit through the transmission path, 

at least one of the first unit and the second unit being 
15 for handling at least one of video, audio, and information, 
the device 

having screen display data for displaying an operating 
screen of the device and overlap display data for overlap display 
on the operating screen; and 
20 transmitting the screen display data and the overlap 

display data to the controller through the transmission path, and 
the controller 

comprising a user interface including display means; 
receiving the screen display data and the overlap display 
25 data from the device through the transmission path; 



displaying the operating screen on the display means using 
the screen display data, and carrying out overlap display on the 
operating screen displayed on the display means by using the 
overlap display data; and 
5 in response to an operation by a user to the operating 

screen, controlling the device by transmitting operation 
information indicative of the operation to the device through the 
transmission path. 

According to a fourteenth aspect of the present invention, 
10 in the thirteenth aspect, 

the overlap display data is cursor information for 
displaying a cursor indicative of a position of operation by the 
user to said operating screen. 

According to a fifteenth aspect of the present invention, 
15 in the fourteenth aspect, 

the cursor information includes position information 
indicative of a position of the cursor on the operating screen. 

According to a sixteenth aspect of the present invention, 
in the fourteenth aspect, 
20 the cursor information includes shape information 

indicative of a shape of the cursor. 

According to the sixteenth aspect of the present invention, 
with cursor shape information being set, it is possible to conform 
the design of the cursor to the design of each display element 
25 on the operating screen. 



According to a seventeenth aspect of the present invention, 
in the fourteenth aspect, 

the cursor information includes size information 
indicative of a size of the cursor. 
5 According to the seventeenth aspect of the present 

invention, with cursor size information being set, it is possible 
to conform the size of the cursor to the size of each display 
element on the operating screen. 

According to an eighteenth aspect of the present invention, 
10 in the fourteenth aspect, 

the cursor information includes color information 
indicative of a color of the cursor. 

According to the eighteenth aspect of the present invention, 
with cursor color information being set, it is possible to conform 
15 the design of the cursor to the design of each display element 
on the operating screen. Furthermore, according to the 
eighteenth aspect, the GUI can be represented as supposed in the 
device, and thereby a GUI creator can easily convey his/her 
intention to users . 
20 According to a nineteenth aspect of the present invention, 

in the fourteenth aspect, 

the cursor information includes enable information 
indicating an operation that the user is allowed to perform. 

According to the nineteenth aspect of the present invention, 
25 when an operator carries out operation, whether the operation is 

15 



allowed or not is determined using enable information in the 
cursor information. Therefore, if the prohibited operation is 
instructed, operation information is not transmitted to the 
device, and error handling can be made in the controller . 
5 According to a twentieth aspect of the present invention, 

in the thirteenth aspect, 

when a display part corresponding to the overlap display 
data of the device is changed, the device transmits the overlap 
display data to the controller, and 

10 the controller receives the overlap display data from the 

device through the transmission path and, based on the received 
overlap display data, updates the operating screen displayed on 
the display means . 

According to the twentieth aspect of the present invention, 

15 if only the overlap display data is changed due to an operation 
from the user, the device that has received the operation 
information indicative of that operation sends overlap display 
data to the controller, but does not send the screen display data 
thereto. Using the received overlap display data, the controller 

20 updates overlap display on the operating screen. Therefore, when 
the cursor is displayed using the overlap display data, for 
example, the data amount to be transferred is small, thereby 
improving a response speed to cursor move operation. 

A twenty-first aspect of the present invention is directed 

25 to a network control system in which a first unit and a second 



unit are connected to each other a transmission path and a 
controller included in the first unit controls a device in the 
second unit through the transmission path, 

at least one of the first unit and the second unit being 
5 for handling at least one of video, audio, and information, 
the device 

having screen display data for displaying an operating 
screen of the device and identification information for 
identifying the screen display data; and 
10 transmitting the screen display data and the 

identification information to the controller through the 
transmission path, and 
the controller 

comprising a user interface including display means; 
15 receiving the screen display data and the identification 

information from the device through the transmission path; and 
displaying the operating screen on the display means using 
the screen display data. 

A twenty- second aspect of the present invention is directed 
20 to a network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in the first unit controls a device in the 
second unit through the transmission path, 

at least one of the first unit and the second unit being 
25 for handling at least one of video, audio, and information. 



the device 

having screen display data for displaying an operating 
screen of the device and identification information for 
identifying the screen display data; 
5 transmitting the screen display data and the 

identification information to the controller through the 
transmission path, and 
the controller 

comprising a user interface including display means; 
10 receiving the screen display data and the identification 

information from the device through the transmission path; and 
displaying the operating screen on the display means using 
the screen display data and the identification information. 

A twenty- third aspect of the present invention is directed 
15 to a network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in the first unit controls a device in the 
second unit through the transmission path, 

at least one of the first unit and the second unit being 
20 for handling at least one of video, audio, and information, 
the device 

having screen display data for displaying an operating 
screen of the device and overlap display data for overlap display 
on the operating screen; and 
25 transmitting the screen display data and the overlap 



display data to the controller through the transmission path, and 
the controller 

comprising a user interface including display means; 
receiving the screen display data and the overlap display 
5 data from the device through the transmission path; and 

displaying the operating screen on the display means using 
the screen display data, and carrying out overlap display on the 
operating screen displayed on the display means by using the 
overlap display data. 
10 A twenty-fourth aspect of the present invention is directed 

to a second unit that is connected to each other a transmission 
path to a first unit including a controller and includes a device 
controlled by the controller through the transmission path, 

at least one of the first unit and the second unit being 
15 for handling at least one of video, audio, and information, 
the device 

having screen display data for displaying an operating 
screen of the device and identification information for 
identifying the screen display data; 
20 transmitting the screen display data and the 

identification information to the controller through the 
transmission path; and 

receiving the identification information of the screen 
display data and operation information indicative of an operation 
25 by a user, and operating based on the received identification 



information and operation information. 

A twenty-fifth aspect of the present invention is directed, 
to a first unit that is connected to each other a transmission 
path to a second unit and includes a controller for controlling 
5 a device included in the second unit through the transmission 
path, 

at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, 
the controller 

10 comprising a user interface including display means; 

receiving screen display data indicative of an operating 
screen of the device and identification information for 
identifying the screen display data from the device through the 
transmission path; 
15 displaying the operating screen on the display means using 

the screen display data; and 

in response to an operation by a user to the operating 
screen, controlling the device by transmitting operation 
information indicative of the operation and the identification 
20 information to the device through the transmission path. 

According to a twenty- sixth aspect of the present invention, 
in the twenty-fourth aspect, 

the operation information includes operating position 
information indicative of a position of operation on the operating 
25 screen. 



According to a twenty- seventh aspect of the present 
invention, in the twenty- fifth aspect, 

the operation information includes operating position 
information indicative of a position of operation on the operating 
5 screen. 

A twenty-eighth aspect of the present invention is directed 
to a second unit that is connected to each other a transmission 
path to a first unit including a controller and includes a device 
controlled by the controller through the transmission path, 
10 at least one of the first unit and the second unit being 

for handling at least one of video, audio, and information, 
the device 

having screen display data composed of a plurality of 
partial screen display data for displaying an operating screen 
15 of the device; 

transmitting the partial screen display data to the 
controller through the transmission path; and 

receiving operation information indicative of an 
operation by a user, and operating based on the received operation 
20 information, 

A twenty-ninth aspect of the present invention is directed 
to a first unit that is connected to each other a transmission 
path to a second unit and includes a controller for controlling 
a device included in the second unit through the transmission 
25 path, 
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at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, 
the controller 

comprising a user interface including display means; 
5 receiving a plurality of partial screen display data 

indicative of an operating screen of the device through the 
transmission path from the device; 

displaying the operating screen on the display means using 
the partial screen display data; and 
10 in response to an operation by a user to the operating 

screen, controlling the device by transmitting operation 
information indicative of the operation through the transmission 
path to the device. 

According to a thirtieth aspect of the present invention, 
15 in the twenty-eighth aspect, 

when the screen display data of the device is changed, the 
device transmits changed partial screen display data of the screen 
display data to the controller. 

According to a thirty- first aspect of the present invention, 
20 in the twenty-ninth aspect, 

when the screen display data of the device is changed, the 
controller receives changed partial screen display data of the 
screen display data from the device through the transmission path 
and, based on the received partial screen display data, updates 
25 the operating screen displayed on the display means . 



A thirty- second aspect of the present invention is directed 
to a second unit that is connected to a transmission path to a 
first unit including a controller and includes a device controlled 
by the controller through the transmission path, 
5 at least one of the first unit and the second unit being 

for handling at least one of video, audio, and information, and 
the device 

having screen display data for displaying an operating 
screen of the device and overlap display data for overlap display 
10 on the operating screen; 

transmitting the screen display data and the overlap 
display data to the controller through the transmission path; and 
receiving operation information indicating a user's 
operation transmitted from the controller, and operating based 
15 on the received operation information. 

A thirty- third aspect of the present invention is directed 
to a first unit that is connected through a transmission path to 
a second unit and includes a controller for controlling a device 
included in the second unit through the transmission path, 
20 at least one of the first unit and the second unit being 

for handling at least one of video, audio, and information, 
the controller 

comprising a user interface including display means; 
receiving screen display data indicative of an operating 
25 screen of the device and overlap display data for overlap display 
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on the operating screen through the transmission path; 

displaying the operating screen of the device on the 
display means using the screen display data, and performing 
overlap display on the operating screen displayed on the display 
5 means using the overlap display data; and 

in response to an operation by a user to the operating 
screen, controlling the device by transmitting operation 
information indicating the operation through the transmission 
path to the device. 
10 A thirty- fourth aspect of the present invention is directed 

to a control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
path, at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, and 
15 a controller included in the first unit controlling a device 
included in the second unit through the transmission path, the 
control method comprising the steps of : 

transmitting screen display data for displaying an 
operating screen of the device and identification information for 
20 identifying the screen display data from the device through the 
transmission path to the controller; 

displaying the operating screen on the controller using the 
screen display data transmitted from the device; and 

in response to an operation by a user to the operating screen, 
25 controlling the device by transmitting operation information 



indicative of the operation and the identification information 
from the controller through the transmission path to the device. 

A thirty-fifth aspect of the present invention is directed 
to a control method, in a network control system in which a first 
5 unit and a second unit are connected to each other a transmission 
path, at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, and 
a controller included in the first unit controlling a device 
included in the second unit through the transmission path, the 

10 control method comprising the steps of : 

transmitting screen display data for displaying an 
operating screen of the device and identification information for 
identifying the screen display data from the device through the 
transmission path to the controller; 

15 displaying the operating screen on the controller using the 

screen display data and the identification information 
transmitted from the device; and 

in response to an operation by a user to the operating screen, 
controlling the device by transmitting operation information 

20 indicative of the operation and the identification information 
from the controller through the transmission path to the device. 

A thirty- sixth aspect of the present invention is directed 
to a control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 

25 path, at least one of the first unit and the second unit being 



for handling at least one of video, audio, and information, and 
a controller included in the first unit controlling a device 
included in the second unit through the transmission path, the 
control method comprising the steps of : 
5 transmitting at least one of plurality of partial screen 

display data composing screen display data for displaying an 
operating screen of the device from the device through the 
transmission path to the controller; 

displaying the operating screen on the controller using the 

10 partial screen display data transmitted from the device; and 
in response to an operation by a user to the operating screen, 
controlling the device by transmitting operation information 
indicative of the operation from the controller through the 
transmission path to the device. 

15 A thirty- seventh aspect of the present invention is 

directed to a control method, in a network control system in which 
a first unit and a second unit are connected to each other a 
transmission path, at least one of the first unit and the second 
unit being for handling at least one of video, audio, and 

20 information, and a controller included in the first unit 
controlling a device included in the second unit through the 
transmission path, the control method comprising the steps of: 
transmitting screen display data for displaying an 
operating screen of the device and overlap display data for 

25 overlap display on the operating screen from the device through 



the transmission path to the controller; 

displaying the operating screen on the controller and 
performing overlap display on the displayed operating screen 
based on the overlap display data by using the screen display data 
5 and the overlap display data transmitted from the device; and 
in response to an operation by a user to the operating screen, 
controlling the device by transmitting operation information 
indicative of the operation from the controller through the 
transmission path to the device. 
10 A thirty- eighth aspect of the present invention is directed 

to a control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
path, at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, and 
15 a controller included in the first unit controlling a device 
included in the second unit through the transmission path, said 
control method comprising the steps of: 

transmitting screen display data for displaying an 
operating screen of the device and identification information for 
20 identifying the screen display data from the device through the 
transmission path to the controller; and 

displaying the operating screen on the controller using the 
screen display data transmitted from the device. 

A thirty-ninth aspect of the present invention is directed 
25 to a control method, in a network control system in which a first 



unit and. a second unit are connected to each other a transmission 
path, at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, and 
a controller included in the first unit controlling a device 
5 included in the second unit through the transmission path, the 
control method comprising the steps of: 

transmitting screen display data for displaying an 
operating screen of the device and identification information for 
identifying the screen display data from the device through the 

10 transmission path to the controller; and 

displaying the operating screen on the controller using the 
screen display data and the identification information 
transmitted from the device. 

A fortieth aspect of the present invention is directed to 

15 a control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
path, at least one of the first unit and the second unit being 
for handling at least one of video, audio, and information, and 
a controller included in the first unit controlling a device 

20 included in the second unit through the transmission path, the 
control method comprising the steps of : 

transmitting screen display data for displaying an 
operating screen of the device and overlap display data for 
overlap display on the operating screen from the device through 

25 the transmission path to the controller; and 



displaying the operating screen on the controller and 
performing overlap display on the operating screen based on the 
overlap display data by using the screen display data and the 
overlap display data transmitted from the device. 

5 

BRIEF DESCRIPTION OF THE DRAWINGS 
FIG. 1 is a block diagram showing the configuration of a 
device in a network control system according to a first embodiment 
of the present invention. 
10 FIG. 2 is a block diagram showing the configuration of a 

controller in the network control system according to the first 
embodiment . 

FIG. 3 is a diagram showing one example of system 
configuration of the network control system in the first 
15 embodiment . 

FIG. 4 is a diagram showing the structure of screen 
information sent from a device to a controller in the first 
embodiment . 

FIG. 5 is a diagram illustrating one protocol of the network 
20 control system in the first embodiment. 

FIG. 6 is a diagram showing one example of an operating 
screen displayed on the controller in the first embodiment. 

FIG. 7 is a diagram illustrating another protocol of the 
network control system in the first embodiment . 
25 FIG. 8 is a diagram showing the structure of screen 



information sent from a device to a controller in a second 
embodiment of the present invention. 

FIG. 9 is a diagram showing one example of an operating 
screen displayed on the controller in the second embodiment. 
5 FIG. 10 is a block diagram showing the configuration of a 

version manager of the device in the second embodiment. 

FIG. 11 is a flowchart showing the operation of the version 
manager of the device in the second embodiment . 

FIG. 12 is a diagram illustrating version information in 
10 the second embodiment . 

FIG. 13 is a diagram illustrating a protocol of a network 
control system in the second embodiment. 

FIG. 14 is a diagram showing one example of an operating 
screen displayed on the controller in a third embodiment of the 
15 present invention. 

FIG. 15 is a diagram illustrating a protocol of a network 
control system in the third embodiment . 

FIG- 16 is a diagram showing the structure of screen 
information sent from a device to a controller in a fourth 
20 embodiment of the present invention. 

FIG. 17 is a diagram showing one example of an operating 
screen displayed on the controller in the fourth embodiment. 

FIG. 18 is a diagram showing one example of an operating 
screen displayed on the controller in a modified fourth 
25 embodiment . 



FIG. 19 is a diagram showing the structure of screen 
information sent from a device to a controller in a fifth 
embodiment of the present invention. 

FIG. 20 is a diagram showing one example of an operating 
5 screen displayed on the controller in the fifth embodiment . 

BEST MODE FOR CARRYING OUT THE INVENTION 
The present invention is now described in more detail 
referring to the attached drawings . 
10 < First embodiment > 

FIG. 1 is a block diagram showing the configuration of a 
device in a network control system according to a first embodiment 
of the present invention, and FIG. 2 is a block diagram showing 
the configuration of a controller in the network control system 
15 according to the first embodiment . The operation and 
configuration of the network system of the present embodiment are 
now described by using FIGS. 1 and 2. 

A device described herein means to be controlled by a 
controller. The controller herein means that it has a user 
20 interface including display means such as a video monitor having 
a graphic display function and operating means such as a mouse, 
keyboard, and TV remote and controller that a user operates while 
watching the screen of the video monitor. Note that either one 
or both of the device and the controller may exist in a single 
25 unit. Further, the device corresponds to one node on a 
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transmission path, and a unit may be constructed so as to have 
a plurality of nodes in a single box. The controller and the video 
monitor may be different units, and the connection therebetween 
is arbitrarily determined: they may be connected in an analog 
5 manner or with a transmission path as shown in the present 
embodiment . 

<1.1 Device in the first embodiment > 

As shown in FIG. 1, the device in the network control system 
of the present embodiment includes a packet transmitter/receiver 

10 2 , a synchronous data transmitter/receiver 3 , a device signal 
processing part 4, an asynchronous data transmitter/receiver 5, 
a device asynchronous data processing part 6 , a unit configuration 
information 7, an information manager 18, and a screen display 
data generator 20. The packet transmitter/receiver 2 is 

15 connected to a transmission path 1. 

The transmission path 1 is, by way of example only, a serial 
bus defined by IEEE 1394 standard (IEEE 1394-1995 and standards 
upward -compatible therewith) (1394 bus). Note that the 
transmission path 1 is not necessarily the 1394 bus, but is 

20 implemented by using ATM, Ethernet, infrared transmission, or the 
like. 

The packet transmitter /receiver 2 physically and 
electrically interfaces with the transmission path 1, and also 
carries out arbitration of bus mastership, cycle control for 
25 synchronous transfer, and other processing. Further, the packet 



transmitter /receiver 2 selectively receives a packet on the 
transmission path 1 according to the destination and transmits 
the packet to the transmission path 1 . 

At transmission, the synchronous data transmitter /receiver 

5 3 manages the transfer rate (data dividing) and provides a header. 
For example, when the 1394 bus of the AV protocol ( lEC 61883) 
standard is used, the synchronous data transmitter/receiver 3 
provides a CIP (Common Isochronous Packet) header. On the other 
hand, when receiving data, the synchronous data 

10 transmitter /receiver 3 rearranges received packets in a correct 
order, removes the headers, and performs other processing. 

The device signal processing part 4 receives synchronous 
data from the synchronous data transmitter/receiver 3 to carry 
out signal processing according to the device. For example, if 

15 the device is a recorder/ player such as a digital VCR, the device 
signal processing part 4 records the synchronous data into a 
recording medium (magnetic tape, for example) . In addition, the 
device signal processing part 4 retrieves the synchronous data 
from the recording medium, broadcasting wave , etc., and transmits 

20 the same to the synchronous data transmitter/receiver 3. 

The asynchronous data transmitter/receiver 5 carries out 
asynchronous data transactions according to the protocol of the 
transmission path 1. By way of example, for the 1394 bus, read, 
write, lock transactions are performed. Here, the asynchronous 

25 data transmitter/receiver 5 may be implemented as software. 



The device asynchronous data processing part 6 processes 
the asynchronous data received from the asynchronous data 
transmitter/receiver 5, and then transfers the same to the 
appropriate component in the device. For example, if receiving 

5 a control data and user operation information as the asynchronous 
data, the device asynchronous data processing part 6 determines 
validity of the asynchronous data, and instructs a unit internal 
controller 9 to execute functions corresponding to the control 
code and user operation information if the data is valid. Here, 

10 the device asynchronous data processing part 6 may transfer the 
control code and the user operation information to the information 
manager 18. In this case, the information manager 18 determines 
validity of these and, if valid, instructs the unit internal 
controller 9 to execute functions indicated by these. 

15 If the controller requests for the unit configuration 

information 7, the device asynchronous data processing part 6 
sends, according to the request received through the asynchronous 
data transmitter /receiver 5, etc., the information in the unit 
configuration information 7 through the asynchronous data 

20 transmitter/receiver 5, etc., to the controller. Furthermore, 
based on the instruction from the unit internal controller 9, the 
device asynchronous data processing part 6 sends asynchronous 
data from the components in the device to the asynchronous data 
transmitter/receiver 5. Here, the asynchronous data 

25 transmitter /receiver 5 and the device asynchronous data 



processing part 6 may be constructed as a single unit. 

The unit configuration information 7 indicates information 
about the configuration of the device, being written according 
to the rule shown by a configuration ROM of a CSR (Command and 

5 Status Registers) architecture indicated by ISO/IEC 13213:1994. 
If the 1394 bus is used, the device configuration information 
includes bus information such as whether the device supports a 
bus manager and isochronous operation , unit directory including 
information whether the device supports an AV protocol and a 

10 unique ID as an identifier of the device. 

When transmitting bitmap data indicative of contents to be 
displayed on the screen of the display means of the controller 
as a device operating screen (hereinafter referred to as "screen 
display data") to the controller, the information manager 18 

15 provides the screen display data with identification information 
for identifying the screen display data. Further, the 
information manager 18 instructs the screen display data 
generator 20 to update the screen display data according to the 
instruction from the unit internal controller 9. 

20 This identification information includes information for 

specifying the device (unique ID to the device, for example) and 
version information. Byway of example, when the identification 
information includes version information, the information 
manager 18 is constructed using a counter to manage the version 

25 of the screen display data. The counter in the information 



manager 18 is incremented every time the screen information is 
updated by the unit internal controller 9. Here, the counter is 
an infinite cyclic counter having a sufficient finite bit length, 
and indicates a minimum value after its maximum value is 
5 incremented. The bit length of the counter is arbitrarily 
determined. Preferably, however, the bit length is sufficient 
enough to prevent one cycle of the counter value from being 
complete at least while the controller holds control or has access 
to a change of state in order to avoid the same counter value from 

10 indicating different screen display data. 

Further, when the screen display data is read to the 
controller, the counter value is also read thereto. The counter 
value indicates the version information of the read screen display 
data. Thus, the controller can confirm the version information, 

15 thereby improving reliability. 

The screen display data generator 20 generates screen 
display data ( screen display data in a bitmap format , for example) 
indicating the operating screen for the device according to the 
instruction from the unit internal controller 9 based on the state 

20 of the device and user operation. Here, the screen display data 
generated by the screen display data generator 20 is screen 
display data displayed on one screen of the controller. The size 
of the operating screen is predetermined through the 
communication between the device and the controller, standards, 

25 or the like, or written in the header of the screen information. 



etc- On the operating screen displayed using the screen display 
data, control information for controlling the device, contents 
information, operating state information, etc., are displayed. 
Here, the control information indicates an operating panel 

5 (reverse button, stop button, record button, etc. , ) of a VCR, for 
example. The contents information indicates information for 
notifying the user of the contents. For an STB (set-top box), 
the contents information indicates information about a program 
now being broadcast, that is, a program title, title image, theme 

10 music, summary, cast, etc. For a DVD device, the contents 
information indicates the contents recorded on a disk as a 
recording medium, that is, a title, title image, theme music, 
summary, cast, etc. For the VCR, by way of example only, the 
operating state information indicates a display element (also 

15 called "object") such as display elements indicating the state 
of operation of the device (playing, reversing, unattended 
recording programmed). Further, information required for 
network control such as identification information of the 
controller that is now using the device, and time, data, and a 

20 channel number for unattended recording, etc. , may be written in 
the operating state information. 

The unit internal controller 9 controls each components 
including the internal mechanism of the device, etc. If the 
device asynchronous data processing part 6 receives control code 

25 indicating an operation of the device, the unit internal 



controller 9 causes the device to operate according to the control 
code as instructed by the device asynchronous data processing part 
6. Also, according to a change of state in the device and the 
user operation, the unit internal controller 9 instructs the 

5 screen display data generator 20 to update the screen display data 
through the information manager 18. 

The operation of the device in response to the request from 
the controller, etc. , is as follows. If the device or controller 
is connected to the transmission path 1, the controller first 

10 reads the unit configuration information 7 of the device to 
determine whether the device has screen display data or not, 
making a request to the device for GUI information and reading 
the screen display data from the screen display data generator 
20. 

15 Then, if receiving the user operation information from the 

controller, the device carries out processing indicated by the 
operation information according to the circumstances. 

Here, for example, when the controller transmits operating 
information indicating that the user performs "select" operation 

20 (hereinafter referred to as "operation information "select"") 
while displaying the operating screen for the device using the 
screen display data, the asynchronous data transmitter /receiver 
5 reports the operation to the unit internal controller 9 . The 
unit internal controller 9 provide an instruction to each 

25 component in the device according to the state of the screen 



display data. By way of example, assume that screen display data 
indicating that the cursor is displayed on the play button is 
transmitted from the device to the controller and the user 
performs "select" operation while the controller displays the 

5 screen display data. In this case, the device receives the user 
operation information "select", and the unit internal controller 
9 recognizes, from the screen display data or the state of the 
device that has generated the screen display data, that the user 
operation indicates "play" , and provides an instruction to each 

10 components in the device to start the play operation. 

Note that, at this time, in addition to the user operation 
information, the identification information of the screen display 
data being displayed by the controller when the user performed 
operation is transmitted to the device. Therefore, the device 

15 can correctly determine to which screen display data the user 
operation is directed . Such determination can be easily made even 
if the user operation becomes invalid or indicates control of 
another function due to a change of state in the device (for example ; 
when the screen display data is totally changed) . Thus erroneous 

20 operation of the device can be prevented. As such, if the user 
operation becomes invalid due to a change of information in the 
device, the device can update the screen display data and notify 
the user that the operation becomes invalid. 

As described above, in response to the request from the 

25 controller for GUI (Graphical User Interface) information, all 



the device has to do is to provide the screen display data 
information, thereby lightening the load on the device. Further, 
it is not necessary to define a coiranand to each function of the 
device by standardization organizations etc. Even for a device 

5 having a new function that cannot be presently assumed, if screen 
display data compliant with the new function is sent to the 
controller, the new function can be easily presented to the user 
via the transmission path 1, and become available to the user. 
Then, by specifying the function of the device using the user 

10 operation information and the screen display data, the user can 
control the device with a simple structure. 

Note that the component such as the synchronous data 
transmitter/receiver 3 and the device signal processing part 4 
may be arbitrarily implemented according to the function of the 

15 device, and may be omitted. Furthermore, such component may be 
implemented as either hardware or software. 

Further, the user operation information may be encoded and 
then sent. Moreover, the screen information and the user 
operation information are not required to be sent or received 

20 through the same transmission path. By way of example, the screen 
information is transmitted through a 1394 bus using a cable while 
the operation information by infrared rays . 
<1.2 The controller in the first embodiment > 

FIG. 2 is a block diagram showing the configuration of the 

25 controller in the network control system of the present embodiment . 



The controller includes a controller signal processing part 10, 
a controller asynchronous data processing part 11, a screen 
display data manager 12, a display/function selection part 14, 
and a screen display data holding part 21. Note that, in FIG. 
5 2, the same components as those shown in FIG. 1 are provided with 
the same reference numerals, and their description is omitted 
herein . 

The controller signal processing part 10 receives 
synchronous data from the synchronous data transmitter/receiver 

10 3, and carries out signal processing that is capable of being 
performed by the controller. By way of example, if the controller 
is capable of displaying video such as a video monitor, the 
controller signal processing part 10 decodes the synchronous data 
(MPEG2 stream, for example) for display on the screen. 

15 The controller asynchronous data processing part 11 

processes asynchronous data received from the asynchronous data 
transmitter/ receiver 5, and then transmits the processed data to 
each appropriate component in the controller. Furthermore, the 
controller asynchronous data processing part 11 receives, from 

20 the packet transmitter/receiver 2 through the asynchronous data 
transmitter /receiver 5, information about the device on the 
transmission path 1 such as connection of a new device and removal 
of an existing device, and screen display data of the device, and 
then transmits them to the screen display data manager 12. 

25 Further, according to the instruction from the display/function 



selection part 14, the controller asynchronous data processing 
part 11 sends asynchronous data from each component in the 
controller to the asynchronous data transmitter/receiver 5 . 
Here, the asynchronous data transmitter/receiver 5 and the 
5 controller asynchronous data processing part 11 may be 
constructed as a single unit . 

Assume the controller and the device are constructed in a 
single unit . Since the screen display data of the device in the 
unit is generated in the unit internal controller 9 , the unit has 
10 the screen display data of the device but the screen display data 
manager 12 does not necessarily required to manage it. Also, in 
this case, the controller signal processing part 10 and the device 
signal processing part 4, and the controller asynchronous data 
processing part 11 and the device asynchronous data processing 
15 part 6 may be constructed as a single unit, respectively. 

The screen display data manager 12 manages screen display 
data information received from the device on the transmission path 
1. When receiving, from the controller asynchronous data 
processing part 11, information about new device connection, the 
20 screen display data manager 12 instructs the controller 
asynchronous data processing part 11 to read screen display data 
information of the new device. Note that, in this case, the new 
device may search for the controller to spontaneously transmit 
its screen display data to the controller. 
25 Next, when the screen display data and its version 



information of the new device has been read, the screen display 
data is held in the screen display data holding part 21, and its 
identification information is stored in the screen display data 
manager 12 in relation to the screen display data. Here, the 
identification information includes the read version information. 
The identification information may be stored in the screen display 
data holding part 21 together with the screen display data, or 
may be stored and managed by the screen display data manager 12. 
Furthermore, when, for example, information about removal of an 
existing device on the transmission path 1 is received, the 
corresponding screen display data is deleted from the screen 
display data holding part 21. Note that the screen display data 
in the controller does not have to be exactly the same in format 
as that in the device as long as they include the same information. 

The display/function selection part 14 notifies the user 
of the screen display data including a display element 
(video/ audio/ character information, etc.,) indicating a 
function or state of the device on the screen of the controller 
and, according to user operation, selects a device and function, 
and provides instructions of executing each function. This 
display/function selection part 14 instructs the screen display 
data manager 12 to search for the screen display data and, using 
the screen display data obtained through search, causes the 
operating screen for an arbitrarily device on the transmission 
path 1 to be displayed on the screen of the controller. Further, 
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the display/ function selection part 14 can display/replay data 
received from the controller signal processing part 10 (video and 
audio data, for example) and data received from the controller 
asynchronous data processing part 11. At this time, the screen 

5 display data may be overlaid on the video data received from the 
controller signal processing part 10, or the display screens for 
the screen display data and for the video data may be switched 
according to a user's instruction, etc. Further, the background 
of the screen display data may be transparent to allow display 

10 of the video data. 

If the user performs, onto the screen displayed by the 
controller, operation such as pressing a selection button on the 
remote controller and pressing a right -direction arrow button, 
the display/ function selection part 14 issues the identification 

15 information of the screen display data and operation information 
indicative of this user operation through the controller 
asynchronous data processing part 11, etc. 

The display/function selection part 14 receives, as updated 
screen display data, a response from the device to the 

20 identification information and the user operation information 
through the controller asynchronous data processing part 11 . The 
identification information is managed by the screen display data 
manager 12, and the corresponding screen display data in the 
screen display data holding part 21 is updated. Further, if the 

25 identification information added to the screen display data as 



the response includes version information, the screen display 
data manager 12 checks the version information, and updates the 
screen display data held in the screen display data holding part 
21 only when the version information has been correctly updated. 

5 Thus , the screen display data held in the controller does not have 
to be updated every time the same screen display data is 
transmitted, thereby lightening a load. 

In the present embodiment, the controller does not have to 
recognize each function of the device. For example, even for a 

10 device having a new function that cannot be presently assumed, 
the controller can notify the user by displaying screen display 
data including a display element (button, for example) to the new 
function. When, with such display element, the user understands 
and selects the new function, the display/function selection part 

15 14 of the controller transmits this user operation information 
to the device to cause the device to execute the new function. 
Therefore, with the above structure, the user can execute even 
a new function that cannot be presently assumed. 

Note that the components such as the synchronous data 

20 transmitter /receiver 3 and the controller signal processing part 
10 may be omitted according to the function of the controller. 
<1.3 System structure in the first embodiment > 

FIG. 3 is a diagram showing the system structure of the 
network control system of the present embodiment . As shown in 

25 FIG. 3, the network control system includes a television receiver 



(television) 21, a television remote controller 22, a personal 
computer (PC) 23, a recordable/replayable DVD 31, a DV- format 
digital VCR (DVC) 32, a VHS-format digital VCR (DVHS) 33, a 
DV- format digital movie (DVC movie) 34, and a set -top box (STB) 

5 for CS digital broadcasting 35, which are collectively called 
video/audio/information devices. 

These video /audio /information devices are connected to the 
transmission path 1 to form a single network control system (such 
system is called "AVC system" . ) Note that the 

10 video/audio/information devices are not restricted to the above 
mentioned devices, but Include all of existing devices in video, 
audio, and information areas (printer and mini-disc , for example) 
and devices coming in the future. 

In such system of the present embodiment , the television 

15 21 is a unit composed of a controller and a device, and the user 
uses the remote controller 22 to provide instructions to the 
display/function selection part 14. The PC 23 is a unit composed 
of a controller and a device (modem that interfaces with a 
telephone line , video monitor , etc . , ) and the user uses a keyboard , 

20 mouse, etc. to provide instructions to the display/function 
selection part 14 . 

Here, the DVD 31 and the DVC movie 34 are units capable of 
recording and replaying audio and video data (hereinafter 
referred to "AV data"). Each of the DVC 32 and the DVHS 33 is 

25 a unit composed of a device capable of recording and replaying 



AV data and a device having a function as a digital broadcasting 
tuner. The STB 35 is a device having a function as a tuner for 
receiving CS digital broadcasting. 

Here, the DVD 31, DVC 32, DVHS 33, DVC movie 34, and STB 

5 35 are all devices. However, each of these units may be 
implemented as a unit including a controller and a device as long 
as such unit can realize, if small at all, an environment enabling 
the user to operate other devices through a liquid crystal panel, 
etc., that is, for example, as long as the unit enables the user 

10 to carry out operation such as selecting functions of other 
devices with a touch panel, remote controller, etc. Each of these 
units may include a processing function as a controller with its 
remote controller. In such case, only display and audio may be 
displayed on a monitor through analog connection, etc. , and the 

15 user may operate the unit through the remote controller while 
watching the displayed screen. At this time, the unit may include 
a controller and a device. 

<1.4 Screen display data in the first embodiment> 

FIGS. 4(a) to (c) are diagrams showing the structure of 

20 screen display information to be transmitted from the device to 
the controller in the present embodiment. FIG. 4(a) shows screen 
information using an ID as the identification information; FIG. 
4(b) shows screen information whose identification information 
includes version information; and FIG. 4(c) shows screen 

25 information using version information as the identification 



information. 

First, the screen information shown in FIG. 4(a) has an 
identifier (ID) for identifying each screen display data. As such, 
by identifying the screen display data with the identifier, the 
5 device can easily and reliably recognize to which screen display 
data the user operation information from the controller to the 
device is directed, thereby preventing erroneous operation of the 
device. 

The screen information shown in FIG. 4 (b) has, as the 

10 identification information, an identifier (ID) for identifying 
each screen display data and version information. For example, 
if the device is a VCR and each of a deck part and a tuner part 
has a different operating screen, the device can be constructed 
as such that each operating screen is provided with an identifier, 

15 and version information according to changes in each operating 
screen. In this way, if the screen display data is identified 
by the identifier and the version information, the device can 
manage by classification and reliably recognize to which screen 
display data the user operation information from the controller 

20 to the device is directed, thereby preventing erroneous operation 
of the device. Furthermore, the device can easily manage the 
screen display data. Still further, even if the device resent 
the same screen display data due to problems in the transmission 
path, etc. , the controller can easily know the fact and does not 

25 have to carry out unnecessary screen update, thereby reducing the 



processing. 

The screen information shown in FIG. 4(c) does not have an 
identifier ( ID) for identifying each screen display data, but uses 
version information instead as the identification information. 

5 As such, by using a value which sequentially changes according 
to the changes of the screen display data as the identification 
information, the controller and the device can easily manage to 
which screen display data the user operation information from the 
controller to the device is directed. The device can thus easily 

10 know which screen display data the controller is now displaying 
and how much difference there is between the screen on the 
controller and the screen display data of the device. Further, 
if the screen display data transmitted from the device is missing 
or required to be resent due to problems on the transmission path, 

15 etc., the controller can easily know the fact of missing or a 
retransmission cycle to change the processing according to the 
degree of missing or control the screen display in consideration 
of the retransmission cycle, thereby providing a more usable 
display screen to the user. 

20 Note that the screen information may include in its header 

part type information indicating a type of the screen display data 
(for example, a format of the bitmap data) , attribute information 
indicating whether the screen display data is split screen display 
data or not, or has version information or not, size information 

25 indicating the size of the screen display data, and the like. 



Further, the physical/ logical structure of the screen 
information which was read into the controller does not have to 
be the same in format as that in the device as long as information 
that is valid for the screen display included in the screen 

5 information in the controller is the same as that in the device. 
In other words, the screen information does not have to physically 
include the header as shown in FIGS. 4(a) to (c), etc., as long 
as information such as header is managed by the controller. 
<1.5 Control operation in the first embodiment > 

10 FIG. 5 is a diagram illustrating one protocol of the network 

control system in the first embodiment. The operation for 
controlling the controller and the device is now described below 
using the drawing. 

First, if a device 200 is newly connected to the 

15 transmission path 1, a controller 100 on the transmission path 
1 recognizes the new device 200 through bus reset if the bus is 
a 1394 bus , for example . On the instruction for the screen display 
data manager 12, the controller first issues a screen request 101 
through the transmission path 1 to the device 200. Responding 

20 to the screen request 101, the device 200 sends the screen display 
data and its identification information as screen information 111, 
and the controller 100 then reads the screen information 111. 
Then, the screen display data manager 12 of the controller 100 
reads and registers the screen display data to store it in the 

25 screen display data holding part 21 of the controller 100. 



Here, the structure may be as such that, when the new device 
200 is connected to the transmission path 1, the device 200 
spontaneously transmits the screen display data to the controller 
100. Alternatively, the structure may be as such that, when the 
5 new device 200 is connected to the transmission path 1, the 
controller 100 and the device 200 communicate with each other to 
determine a transmission method of the screen display data and 
the device 200 transmits the screen display data to the controller 
100 by following the method. 
10 The screen display data read into the controller 100 is 

identified by the unique ID for each device 200 and the 
identification information. For example, the screen display 
data has a list structure or directory structure with an entry 
for each device 200. Note that how the screen display data is 
15 held is arbitrarily determined. 

The display/function selection part 14 of the controller 
100 refers to the screen display data in the screen display data 
holding part 21 through the screen display data manager 12, and 
displays the same on the display screen. Here, if the user desires 
20 to display the operating screen for another device 200 that is 
connected to the controller 100, for example, the screen display 
data of the devices 200 that are connected to the controller 100 
are sequentially displayed on the display screen by using a 
display switching button on the remote controller, and then the 
25 screen display data representing the operating screen for the 



desired device 200 is displayed. 

Here, if the controller 100 displays the screen display data 
of a specific device 200, the display/function selection part 14 
makes a request to the screen display data manager 12 for the screen 
display data of that device 200 using the unique ID of the device 
200, etc. The screen display data manager 12 retrieves the screen 
display data of the device 200 from all screen display data in 
the screen display data holding part 21, and then transmits the 
same to the display/function selection part 14. The 
display/function selection part 14 causes the screen display data 
to be displayed on the display screen. 

However, if the device 200 connected to the transmission 
path 1 is included in a device list of the display/function 
selection part 14 listing connected devices, but the screen 
display data of that device 200 is not in the screen display data 
holding part 21, the screen display data manager 12 sends the 
screen request 101 shown in FIG. 5 to the device 200 as a command. 
In response, the device 200 returns the screen information 111. 
The screen response 111 includes the screen display data 
indicative of the operating screen of the device 200, etc., and 
the identification information of this screen display data. The 
identification information includes the identifier (ID), the 
version information, etc. This version information indicates 
the version of the screen display data, and is updated every time 
the screen is updated. 
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Note that the device list is a list that is made in the 
display/function selection part 14 by searching the transmission 
path 1 according to change information about the devices 200 
connected to the transmission path 1 (a bus reset signal, for 
5 example, for reporting addition/removal) and detecting the 
connected devices 200. 

The device 200 may return only the identification 
information in the device 200 as the screen information 111. At 
this time, the controller 100 has to further read the screen 
10 display data body from the device 200 . However, if the controller 
100 has cached the screen display data, for example, in order to 
transmit the same screen as that previously transmitted, the 
device 200 does not have to send the screen display data body to 
the controller 100 , thereby reducing data amount to be transmitted, 
15 lightening a load on the transmission path, and simplifying the 
processing in the controller 100 and the device 200 . 

In this way, the screen display data manager 12 reads the 
screen display data of the device 200 thereinto. Therefore, the 
display/function selection part 14 uses the screen display data 
20 manager 12 to read out the desired screen display data from the 
screen display data holding part in the controller 100, and causes 
the same to be displayed on the screen. Thus, the screen display 
data of the arbitrary device 200 can be displayed on the screen. 

Next , if the user operates a button on the remote controller , 
25 etc., the display/function selection part 14 transmits the type 
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of the operated button, the operation ("press", "release", 
"select" . for example) , and the identification information of the 
screen display data to the device 200. In other words, assuming 
that the remote controller has arrow keys indicating upward, 
5 downward, rightward, and leftward directions , if the user presses 
the rightward direction arrow key, for example, the controller 
100 transmits to the device 200, as an operation request 241, 
operation information ( "rightward arrow key" and "press" ) and the 
identification information of the screen display data obtained 
10 from the device to the device 200. Therefore, the device 200 can 
easily and reliably recognize to which operating screen the 
operation is directed, and therefor can execute correct operation. 
If the device 200 is a pointing device 200 such as a mouse is used 
for the operating screen displayed on the controller 100, the 
15 operating information transmitted to the device 200 includes 
information indicating the operating position of the mouse, etc 
(hereinafter referred to as "operating position information"). 
In this case, the device 200 recognizes, from the transmitted 
identification information and operating information including 
20 the operating position information, which operation has been 
performed to which operating screen, and then executes operation 
corresponding to the recognized results . With such structure as 
controlling the device 200 through operation of the mouse, etc. , 
the cursor can be directly moved to a desired position on the 
25 operating screen, thereby improving operability. 
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Note that only the operation information may be transmitted 
to the device 200 as the operation request 241. In this case, 
although the device 200 cannot reliably recognize to which 
operating screen the operation is directed, the processing can 
5 be simplified and the traffic on the transmission path can be 
reduced. Here, the user operation information may be encoded and 
then transmitted with the identification information. 
Alternatively, each user operation information may be sent as a 
single command (its operand is the control code such as the ID 
10 of the screen display data) . Furthermore, if a button allowing 
only selecting operation to the screen display data is pressed, 
for example, the structure may be as such that only the control 
code indicating this button is transmitted to the device 200, 
thereby simplifying the processing and reducing traffic on the 
15 transmission path 1 . 

Next, if the state in the device 200 is changed to cause 
the screen display data of the device 200 to be changed, the device 
200 spontaneously returns screen information 121. The screen 
information 121 includes the identification information and the 
20 screen display data, and its structure is similar to that shown 
in FIGS. 4(a) to (c) . Note that the device 200 may be structured 
so that the screen information 121 becomes a secondary response 
to the screen request 101. At this time, the controller 100 
receives the secondary response, and then again issues a screen 
25 request. Then, if the screen display data is not changed, a 



primary response to the screen request after the secondary 
response may Include only the identification information without 
the screen information. With such structure, the correspondence 
between the request and command can be clarified, and the 
processing of the device 200 can be simplified. 

The screen information 121 is used, by way of example only, 
for the device 200 to change the display of the operating screen 
into a display indicative of rewinding if automatic fast- 
rewinding has started because a tape comes to the end after replay 
in a VCR. Alternatively, the screen information may be used, by 
way of example only, for changing a still picture on the operating 
screen (from a convex display to a concave display) according to 
the operation such as pressing/releasing a button on the operating 
screen . 

After the screen display data in the controller 100 is 
updated, the screen display data manager 12 instructs the 
display/function selection part 14 to update the screen display. 
The display/function selection part 14 updates the screen 
display. 

Note that, in addition to each request /response , etc.. as 
shown in the present embodiment , other communications may be made 
between the controller 100 and the device 200. For example, 
although not shown, the structure may be as such that an 
acknowledge signal for acknowledging each request and response 
is returned. With such procedure, the controller 100 can always 
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recognize the state of the device 200 . 

FIG. 6 is showing one example of the operating screen 
displayed on the controller in the first embodiment. In this 
example, a screen display 500 represented by the screen display 
data transmitted from the device is displayed as the operating 
screen of the VCR. Displayed on the screen display 500 are a 
display element indicating that the device to be controlled is 
the VCR; two title images as contents information; reverse, play, 
fast-forward, pause, stop, and record buttons as control 
information; and channel up and down buttons. 

In the protocol shown in FIG. 5, the operation information 
and the identification information of the screen display data are 
sent from the controller 100 to the device 200 as the operation 
request. Alternatively, as shown in FIG. 7. the version 
information may be used instead of the identification information 
in the operation request 241. With this version information, the 
device 200 can determined to which operating screen the operation 
is directed, thereby obtaining the effect similar to that in a 
case where the protocol shown in FIG. 5 is used. 

In the present embodiment, assuming that the version 
information of the screen display data is sent from the device 
to the controller as part of the screen information as shown in 
FIGS. 4(b) and (c) , a change of state information of the device 
can be reported to the controller by using the version information, 
and to which screen display data the operation information from 

57 



the controller is directed can be easily determined. Therefore, 
the device can change its own state information at an arbitrary 
time. In other words, even between the time the controller 
detects a change in the state information of the device and the 
time the controller takes in the state information, the device 
may change its own state information. Therefore, the device is 
not required to use a buffer for temporarily storing the change 
in the state information, thereby simplifying the processing and 
reducing its storage area. 

Note that the present invention is not applied only when 
the screen display data of the device is updated, but can also 
be applied when the controller keeps track of arbitrary screen 
display data of the device. By way of example only, the present 
system is effective also for a monitoring system for monitoring 
for intruders with images changed and a system for keeping track 
of the state of a device operating at a remote site. 
<1.6 Effects of the first embodiment > 

As described above, according to the present embodiment, 
the device has screen display data representing an operating 
screen of the device and identification information of the screen 
display data. The controller reads the screen display data and 
the identification information from the device. If the 
identification information includes version information, a 
change of the operating screen information of the device can be 
detected by the identification information. Therefore, the 
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controller can easily detect that the state of the device has been 
changed according to an instruction from another controller or 
a change made spontaneously by the device itself. Also, the 
controller can easily and reliably identify the operation 
information in which the state of the device has been correctly 
reflected. Moreover, the device can easily and reliably 
recognize to which operating screen the operation is directed, 
thereby executing the operation as the user desires . 

Further, according to the present embodiment, the 
controller manages the screen display data representing the 
operating screen for the device, and transmits the operation 
information to the device based on the operation by the user. 
However, the controller does not have to understand the meaning 
of the function of the device or the operation (the relation 
between the operation and the function of the device) . In other 
words, user interface information (screen display data, operation 
information, etc. , ) is managed and processed by the controller, 
while control information directly related to device control 
(information indicating the function of the device, information 
indicating the relation between the function of the device and 
the operation, etc. , ) is by the device. Therefore, according to 
the present embodiment, the processing in the controller is 
reduced. 

<2. Second embodiment > 

Next, a network control system according to a second 

59 



embodiment of the present invention is described. The structures 
of the device and controller in the present embodiment are similar 
to those in the first embodiment, and therefore the same 
components are provided with the same reference numerals, and 
5 their description is omitted herein (refer to FIGS. 1 and 2). 
<2.1 Screen information in the second embodiment> 

FIG. 8 is a diagram showing the structure of screen 
information sent from the device to the controller in the present 
embodiment. In the present embodiment, the screen information 
10 is constructed of a plurality of partial screen information 
corresponding to partial screen display data obtained by dividing 
screen display data indicative of an operating screen into two 
or more. Each partial screen information is in a list format, 
including various header information and attribute information. 
15 However, such partial screen information does not have to be in 
a list structure. Alternatively, each partial screen 
information may be independent, or a screen information set list 
does not have to be required. 

Screen information 53 shown in FIG. 8 is constructed of a 
20 screen information set list 70, and partial screen information 
61, 62, and 63. The structure of a header in the screen 
information is similar to that in the first embodiment. The 
screen information set list 70 is constructed of an identifier 
corresponding to the screen information 53 and a plurality of menu 
25 entries. In each menu entry, an identifier for each partial 

60 



screen display data in the partial screen information 61 to 63 
in the screen information 53 is set. 

Each partial screen information 61 to 63 is constructed of 
a header, version information, arrangement information, and the 

5 partial screen display data. Here, the partial screen display 
data is bitmap data for displaying a partial screen that composes 
part of an operating screen corresponding to the screen 
information 53. The header and version information are similar 
to those in the first embodiment. The arrangement information 

10 is information for specifying where the partial screen display 
data is placed on the operating screen corresponding to the screen 
information 53. 

FIG. 9 is a diagram showing one example of the operating 
screen displayed on the controller in the present embodiment . In 

15 this example, the operating screen is constructed of four partial 
screen displays 501. 502, 503, and 504. Therefore, the screen 
display data representing the operating screen is constructed of 
four partial screen display data corresponding to these four 
partial screen displays . As shown in FIG. 9, in the present 

20 embodiment, the screen display data is divided into a plurality 
of partial screen display data in such a manner that a display 
element indicative of a single meaning to the user is not divided. 
<2.2 Version management in the second embodiment> 

FIG. 10 is a block diagram showing the configuration of a 

25 part in the information manager 18 of the device carrying out 



management of a version of the screen display data (hereinafter 
referred to as "version manager"). The version manager in the 
present embodiment is constructed of a screen display version 
generator 91, an update screen display determination part 92, and 
5 a plurality of partial screen display version information holding 
parts 95, 96, 97, .... 

In the above constructed version manager, the screen 
display version information generator 91 manages the version of 
the screen display data generated by the screen display data 
10 generator 20. The screen display version information generator 
91 is implemented by using a counter, and obtains screen display 
update information from the unit internal controller 9 every time 
the screen display data generated by the screen display data 
generator 20 is changed by the information manager 18 to increment 
15 the counter. The counter is an infinite cyclic counter having 
a sufficient finite bit length, and indicates a minimum value 
after incremented to be the maximum. The bit length of the counter 
is arbitrarily determined. Preferably, however, the bit length 
is sufficient enough to prevent one cycle of the counter value 
20 from being complete at least while the controller holds control 
or has access to a change of state in order to avoid the same counter 
value from indicating different screen display data. 

The update screen display determination part 92 uses the 
screen display update information obtained from the unit internal 
25 controller 9 to detect which partial screen display data has been 
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changed, and caused the partial screen display version 
information holding part corresponding to the partial screen 
display data to store version information of the updated partial 
screen display data. 
5 The first partial screen display version information 

holding part 95 holds version information of any partial screen 
display data composing the screen display data indicative of the 
operating screen for the device. The version information of this 
partial screen display data (hereinafter referred to as "first 
10 partial screen display data") is, for example, version 
information of the partial screen display data in the partial 
screen information 61 shown in FIG. 8, that is, the version 
information constructing the partial screen information 61. 

The second partial screen display version information 
15 holding part 96 holds version information of any partial screen 
display data, but other than the above first partial screen 
display data, composing the screen display data indicative of the 
operating screen for the device. The version information of a 
second partial screen display data is. for example, version 
20 information of the partial screen display data in the partial 
screen information 62 shown in FIG. 8. that is. the version 
information constructing the partial screen information 62. 

The third partial screen display version information 
holding part 97 holds version information of any partial screen 
25 display data, but other than the above first and second partial 
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screen display data, composing the screen display data indicative 
of the operating screen for the device. The version information 
of a third partial screen display data is, for example, version 
information of the partial screen display data in the partial 
5 screen information 63 shown in FIG. 8, that is, the version 
information constructing the partial screen information 63. 

FIG. 11 is a flowchart showing the operation of the version 
manager in the information manager 18 of the device. In the 
version manager, when the device is activated, the screen display 
10 version information generator 91 first initializes the version 
information of all partial screen display data, for example, 
setting the version information to "0" (step S701). Next, the 
version manager waits until the screen display data is changed 
(step S702) . When the screen display data is changed, the screen 
15 display version information generator 91 increments the counter 
to update the version information of the screen display data (step 
S703) . The incremented counter value indicates a screen display 
version number after update. Next, the update screen display 
determination part 92 detects using the screen display update 
20 information which partial screen display data has been changed, 
and then causes the partial screen display version information 
holding part concerned with the changed partial screen display 
data to update the version information thereof. That is, the 
screen display version number is substituted into the version 
25 information of the partial screen display data, which is the 
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counter value incremented of the screen display version 
information generator 91 (step S704). In this way, after the 
version information of the partial screen display data is updated, 
it is determined whether an instruction for ending this series 

5 of processing by the version manager is provided or not. If no, 
the procedure returns to step S702 to repeat the above series of 
processing (steps S702 to S705) . While the device 200 is powered 
on, the procedure repeats the above series of processing. If it 
is determined yes (step S705), the series of processing ends. 

10 By way of example only, when the second partial screen 

display data is changed, the above version manager operates as 
follows. First, the screen display version information 
generator 91 obtains screen display update information indicative 
of the change of the second partial screen display data from the 

15 unit internal controller 9 to increment the counter of the screen 
display version information generator 91, thereby updating the 
version information of the screen display data . The update screen 
display determination part 92 detects by this screen display 
update information that the changed display element belongs to 

20 the second partial screen display data, and causes the second 
partial screen display version information holding part 9 6 to 
update the version information of the partial screen display data. 

FIG. 12 is a diagram illustrating how the screen display 
data and the version information of each partial screen display 

25 data change . Here , assume that the screen display data indicative 
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of the operating screen of the device is constructed of first to 
third partial screen display data. 

At the initial state, all items of the version information 
have been initialized . That is , the screen display version number 
5 and each partial screen display version number are set to "0". 
Here, if a display element of the second partial screen display 
data is changed, the counter of the screen display version 
information generator 91 is incremented, thereby setting the 
screen display version number and a second partial screen display 
10 version number to "1". Then, if a display element of the third 
partial screen display data is changed, the above counter is 
updated as such that the screen display version number and a third 
partial screen display version number are both set to "2". At 
this time, the second partial screen display version number still 
15 indicates "1". Then, if a display element of the second partial 
screen display data is changed again, the above counter is further 
incremented as such that the screen display version number and 
the second partial screen display version number are both set to 
"3". In other words, the second partial screen display version 
20 number has not and will not become " 2 " . 

In this way, if any display element is changed, the counter 
of the screen display version information generator 91 is 
incremented, and the screen display version number and the version 
number of the partial screen display data to which the changed 
25 display element belongs are both updated to take the counter value 
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after increment 

<2.3 Control operation in the second embodiment > 

FIG. 13 is a diagram illustrating a protocol of the network 
control system in the second embodiment. Using the drawing, the 
5 control operation of the controller and device is now described 
below. Note that the operation other than the control operation 
described below is similar to that in the first embodiment. 

When the device 200 is newly connected to the transmission 
path 1, the controller 100 on this transmission path 1 issues, 
10 like the first embodiment , the screen request 101 to the new device 
200. In response, the new device 200 sends screen information 
111b. and the controller 100 reads the screen information 
thereinto. At this time, the screen information 111b sent from 
the device 200 is the screen information 53 including the screen 
15 information set list 70 shown in FIG. 8. However, without using 
the screen information set list 70, all partial screen information 
may be sent as the screen information 111b. 

Then, when the user carries out operation to the device 200 
through the operating screen for the controller 100 as a user 
20 interface, the controller 100 transmits the identification 
information of the screen display data indicative of the operating 
screen and the operation information indicative of the operation 
to the device 200 as the operation request 241, like in the first 
embodiment. At this time, the identification information may 
25 include version information. 
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Then, if the state in the device 200 is changed and thereby 
the screen display data in the device 200 is changed, the device 
200 spontaneously transmits partial screen information 121b to 
the controller 100- Unlike the screen information 121 
5 transmitted from the device 200 to the controller 100 in the first 
embodiment, this partial screen information 121b is composed of 
the partial screen display data to which a changed display element 
belongs and its identification information. That is, in the 
present embodiment, if the screen display data is changed, the 
10 partial screen display data to which the changed display element 
belongs is transmitted to the controller 100 . In the controller 
100, after receiving this partial screen display information 121b, 
the screen display data manager 12 updates, among the partial 
screen display data held in the screen display data holding part 
15 21, the partial screen display data indicated by the 
identification information of the partial screen information 121b 
with the partial screen display data of the partial screen 
information 121b. In addition to the partial screen display data, 
the version information held in the screen display data holding 
20 part 21 is also updated. 

<2.4 Effects of the second embodiment> 

In the present embodiment, as shown in FIG. 9, the screen 
display data is divided into a plurality of partial screen display 
data in such a manner that a display element indicative of a single 
25 meaning to the user is not divided. Therefore, if only a single 
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element is changed, the plurality of the partial screen display 
data are not required to be sent. Thus, the data amount to be 
transferred to the controller when the screen display is changed 
is reduced, and the processing in the device and the controller 
is simplified. 

Furthermore, according to the present embodiment, an 
identifier is provided for each partial screen display data. If 
the screen display data is changed, the partial screen display 
data to which a changed display element belongs and its identifier 
are sent to the controller . Therefore , the changed partial screen 
display data can be easily specified. Moreover, at this time, 
the version information of the partial screen display data is also 
transmitted to the controller. With this version information, 
the controller can prevent erroneous display. 

Note that a manner of dividing the screen display data may 
varied every time the device changes the screen display data 
according to the change of the information in the device . By way 
of example, the controller always reconstructs the partial screen 
display data transmitted from the device to create a frame of 
screen display data. The created screen display data is stored 
in screen display data holding part 21. Then, the partial screen 
display data updated by the device is superimposed on the screen 
display data held in the screen display data holding part 21 for 
update. With such structure, the device can transmit partial 
screen display data of an arbitrary size at an arbitrary position 
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to the controller, thereby simplifying the transmission amount 
and the processing of the controller and device. Moreover, at 
this time, in response to the user's operation, the controller 
sends the version information of the screen display data together 
with the user operation information to the device. 
<3. Third embodiment > 

Next, a network control system according to a third 
embodiment of the present invention is now described. Since the 
structure of the device and controller in the present embodiment 
is similar to that in the first embodiment, the same components 
are provided with the same reference numerals, and their 
description is omitted herein (refer to FIGS. 1 and 2). 

Furthermore, the structure of the screen information in the 
present embodiment is similar to that in the second embodiment 
(refer to FIG. 8) in that the screen display data indicative of 
the operating screen of the device is composed of a plurality of 
partial screen display data. However, the present embodiment is 
different from the second embodiment in the manner of dividing 
the screen display data into a plurality of partial screen display 
data. 

FIG. 14 is a diagram showing one example of the operating 
screen displayed on the display screen of the controller in the 
present embodiment. In the present embodiment, the partial 
screen display data is generated for one or more display elements 
selected through a single operation by the user, that is , for every 
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operation . The screen display data is constructed of this partial 
screen display data generated in the above described manner, and 
partial screen display data representing a display which is a base 
for the display elements indicated by the generated partial screen 

5 display data. For example, the operating screen shown in FIG. 
14 is constructed of a plurality of partial screen displays 511 
to 520 and partial screen display 510 which is a base for these 
partial screen displays 511 to 520 (hereinafter referred to as 
"base screen display"). Therefore, the screen display data 

10 indicative of the operating screen of FIG. 14 is composed of the 
partial screen display data corresponding to these partial screen 
display 511 to 520. Each display element as an operation unit 
that corresponds to each of the plurality of partial screen 
displays 511 to 520 is superimposed onto the partial screen 

15 display 510, that is, the base screen display for overlap display. 

FIG. 15 is a diagram illustrating a protocol of the network 
control system in the present embodiment. By using the drawing, 
the control operation of the controller and device is now 
described below. Note that the operation other than the control 

20 operation described below is similar to that in the second 
embodiment . 

The controller 100 issues the screen request 101 to the 
device 200. In response, the device 200 transmits the screen 
information 111b to the controller 100. Then, when the user 
25 carries out operation to the device 200 through the user interface 



of the controller 100, the controller 100 issues an operation 
request 241c to the device 200. By issuing the operation request 
241c, the controller 100 sends identification information of the 
partial screen display data to which a display element to be 
5 operated by the user belongs and operation information indicative 
of the operation to the device 200. In other words, as to the 
operation request 241c in the present embodiment. the 
identification information of the partial screen display data 
corresponding to the operation is transmitted to the device 200, 
10 instead of the identification information of the screen display 
data indicative of the operating screen. In this point, the 
present embodiment is different from the second embodiment. In 
the present embodiment, as shown in FIG. 14, each operation unit 
displayed on the operating screen has a one-to-one correspondence 
15 with each partial screen display data. Therefore, by receiving 
the operation request 241b, the device 200 can easily recognize 
the user operation included in the operation request 241b. In 
other words, in the device 200, based on the identification 
information of the partial screen display data and the operation 
20 information received as the operation request 241b, the device 
asynchronous data processing part 6 instructs the unit internal 
controller 9 to execute a function that corresponds to the 
operation unit indicated by the identification information and 
the operation information. The unit internal controller 9 
25 executes the function based on the instruction. 



Then, when the state in the device 200 is changed and the 
screen display data in the device 200 is thereby changed, the 
device 200 spontaneously transmits partial screen information 
121b to the controller 100. The partial screen information 121b 
5 is composed of, similarly to the second embodiment, the partial 
screen display data to which a changed display element belongs 
and its identification information. After receiving the partial 
screen information 121b, the controller 100 updates, similarly 
to the second embodiment, the partial screen display data and its 
10 version information. 

According to the present embodiment as described above, the 
screen display data indicative of the operating screen is divided 
for every operation unit. Therefore, the device 200 can easily 
recognize the user's operation by the identification information 
15 of the partial screen display data transmitted from the controller 
100 as the operation request 241c. Note that, if the partial 
screen data is divided in such a manner that any operation unit 
is not divided, the device 200 can easily recognize the operation 
even if division is made not by each operation unit. This is 
20 because, although the operation itself cannot necessarily be 
specified only with the identification information of the partial 
screen display data, the type of operation can be narrowed down 
with the identification information. 

Further, in the present embodiment, when one display 
25 element is changed, the device 200 transmits to the controller 

73 



100 only the partial screen information corresponding to the 
partial screen display data to which the display element belongs. 
Therefore, as in the second embodiment, the data amount to be 
transferred to the controller 100 with change in screen display 

5 can be reduced. 

Still further, the controller by itself can also make the 
cursor indicative of the operating position to the user displayed. 
Therefore, the controller can manage cursor movement, thereby 
simplifying the processing in the device, reducing the 

10 transmission amount, and improving response to the user's 
operation. 

<4. Fourth embodiment > 

Next, a network control system according to a fourth 
embodiment of the present invention is now described. The 

15 structure of the device and controller in the present embodiment 
is also similar to that in the first embodiment. Therefore, the 
same components are provided with the same reference numerals, 
and their description is omitted herein (refer to FIGS. 1 and 2) . 

FIG. 16 is a diagram showing the structure of screen 

20 information 54 sent from the device to the controller in the fourth 
embodiment of the present invention. The screen information 54 
is constructed of a screen information set list 71, main screen 
information 65, and cursor display element information 80. In 
the present embodiment, a cursor display element indicative of 

25 the operating position of the user is superimposed on the 

74 



operating screen for overlap display. The operating screen 
itself corresponds to the main screen information 65, while the 
cursor display element to the cursor display element information 
80. As shown in FIG. 16, the main screen information 6 5 is 
5 composed of screen display data as bitmap data indicative of the 
operating screen, a header including an identifier of the screen 
display data, and version information of the screen display data. 
The cursor display element information 80 is composed of cursor 
data as bitmap data representing the cursor display element, a 
10 header including an identifier of the cursor data, version 
information of the cursor data, and position information 
indicative of the position of the cursor. The screen information 
set list 71 is composed of a header including an identifier of 
the screen display data indicative of the operating screen, in 
15 a broad sense, including the cursor display element (broad- sense 
screen display data) , a data entry to the main screen information 
65 , and a data entry to the cursor display element . The identifier 
of the screen display data indicative of the operating screen 
itself is set as the data entry to the main screen information 
20 65, while the identifier of the cursor data is set as the data 
entry to the cursor display element . 

FIG. 17 shows one example of the operating screen displayed 
on the display screen of the controller in the present embodiment. 
In this example, a cursor display element 600 is superimposed onto 
25 a screen display 500, which is the operating screen itself, for 
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overlap display. Note that it can be thought that the broad screen 
display data indicative of the broad operating screen that 
includes the cursor display element 600 is composed of two pieces 
of partial screen display data corresponding to the screen display 
5 500 and the cursor display element 600. Furthermore, the cursor 
position on the operating screen is represented by coordinate 
values (X and Y coordinates) taking an upper- left corner of the 
screen display 500 as the origin. 

The protocol of the network control system in the present 
10 embodiment is basically similar to that in the second embodiment 
as shown in FIG. 13. In the present embodiment, the screen 
information 111b transmitted from the device 200 to the controller 
100 as a response to the screen request 101 is constructed as shown 
in FIG. 16 . The main screen information 65 and the cursor display 
15 element information 80 correspond to different partial screen 
display information in the second embodiment. After receiving 
such screen information, the controller 100 displays the 
operating screen using the screen display data of the main screen 
information 65 and, by using the cursor data and position 
20 information of the cursor display element information 80 , further 
superimposes the cursor display element onto the operating screen 
for overlap display. 

If the controller 100 issues the operation request 241, the 
operation information is transmitted to the device 200 together 
25 with the identification information of the screen display data. 
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The operation information includes the operating position 
information indicative of the operating position of the cursor. 
The operating position information is composed of X and Y 
coordinates on the operating screen. 
5 After issuance of the operation request 241, when the state 

in the device 200 is changed and therefore the screen display data 
in the device 200 is changed, the device 200 spontaneously 
transmits the partial screen information 121b to the controller 
100, like in the second embodiment. At this time, in the present 
10 embodiment , as to cursor movement , only the cursor display element 
information 80 shown in FIG. 16 is sent to the controller 100. 
After receiving, the controller 100 updates overlap display of 
the cursor display element on the operating screen based on the 
position information included in the cursor display element 
15 information 80. Therefore, according to the present embodiment, 
the response speed to cursor move operation is improved. 

Note that the bitmap data of the cursor may be transmitted 
only in the first response (screen information 111b) to the screen 
request, and screen information transmitted thereafter does not 
20 have to include the bitmap data of the cursor. Therefore, the 
processing can be simplified, the transmission amount can be 
reduced, and the response speed can be improved. Furthermore, 
the structure may be as such that the bitmap data indicative of 
the cursor is sent not from the device, but determined by the 
25 controller itself. At this time, as to cursor movement, etc., 
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the device transmits only the updated cursor position information 
as screen information, thereby simplifying the processing, 
reducing the transmission amount, and improving the response 
speed. 

<4.1 Modified example of the fourth embodiment > 

In the above embodiment, the operating screen itself is 
represented by single screen display data. However, as in the 
second embodiment, the operating screen may be represented by a 
plurality of partial display screen data. FIG. 18 is a diagram 
showing one example of display of the operating screen in the 
present embodiment represented by a plurality of partial screen 
display data. In this example, the operating screen is composed 
of four partial screen displays 501, 502, 503, and 504 , and a cursor 
display element 601 is superimposed onto this operating screen 
for display. Therefore, in this case, the main screen information 
65 in the screen information 54 shown in FIG. 16 is replaced by 
a plurality of partial screen display information as shown in FIG. 
8 . Note that the cursor display element information can be 
considered as one type of partial screen display information. 

In this case, the structure of the cursor display element 
information is similar to that of the cursor display element 
information 80 shown in FIG. 16. This information in the cursor 
element information is represented by the coordinates on the 
operating screen in the example shown in FIG. 16. In this case, 
however, as shown in FIG, 18, the arrangement information is 
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information indicating a relative position on the partial screen 
display 502 onto which the cursor display element 601 is 
superimposed for overlap display. That is, the cursor position 
information is represented by coordinate values (relative X and 
Y coordinates) taking the upper- left corner of the partial screen 
display 502 as the origin. 

As described above, the protocol of the network control 
system in a case where the screen display data is composed of a 
plurality of partial screen display data is basically similar to 
that in the third embodiment as shown in FIG. 15. In this case, 
when the user carries out operation through the cursor, the 
controller 100 issues the operation request 241c to the device 
200. At that time, the identification information of the partial 
screen display data sent to the device 200 includes the 
identification information of the cursor data. For example, in 
the case shown in FIG. 18, in addition to the identification 
information of the partial screen display data of the partial 
screen display 502, the identification information of the cursor 
data of the cursor display element 601 is transmitted to the device 
200. The device 200 executes a predetermined function based on 
the identification information of the partial screen display data 
received as the operation request 241c (including the 
identification information of the cursor data) and the operation 
information. 

Then, when the state in the device 200 is changed and 
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therefore the screen display data in the device 200 is changed, 
the device 200 spontaneously transmits the partial screen 
information 121b to the controller 100. When the controller 
lOOissues the operation request 241c based on cursor move 
5 operation by the user, all that is changed on the operating screen 
is the position of the cursor display element. Therefore, only 
the partial screen information 121b including the cursor display 
element information is transmitted to the controller 100. After 
receiving, based on the position information included in the 
10 cursor display element information, the controller 100 
superimposes the cursor display element onto the operating screen 
for overlap display. 
<5, Fifth embodiment > 

Next, a network control system according to a fifth 
15 embodiment of the present invention is now described. The 
structure of the device and controller in the present embodiment 
is also similar to that in the first embodiment. Therefore, the 
same components are provided with the same reference numerals, 
and their description is omitted herein (refer to FIGS. 1 and 2) . 
20 FIG. 19 is a diagram showing the structure of screen 

information 55 sent from the device to the controller in the 
present embodiment . The screen information 55 is constructed of, 
as in the fourth embodiment, the screen information set list 71, 
the main screen information 65, and cursor display element 
25 information 81. In the present embodiment, the cursor display 
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element is superimposed onto the operating screen for overlap 
display. In the screen information 55, the screen information 
set list 71 and the main screen information are similar in internal 
structure to those in the fourth embodiment. The cursor display 

5 element information 81 is similar to that in the fourth embodiment 
in that a header includes an identifier, version information, and 
position information. However, instead of the cursor data as 
bitmap data indicating the cursor display element, the cursor 
display element information 81 has information indicative of the 

10 shape, size, and color of the cursor display element, and further 
operation enable information. In this sense, the present 
embodiment is different from the fourth embodiment . Here , the 
operation enable information is information indicative of the 
type of operation that the user is allowed to carry out. For 

15 example, the operation enable information indicates that the 
cursor can be moved from the present position only to upward and 
rightward directions among upward, downward, leftward, and 
rightward directions . 

FIG. 20 is a diagram showing one example of the operating 

20 screen displayed on the display screen of the controller in the 
present embodiment. In this example, the cursor display element 
601 is superimposed onto the operating screen itself for overlap 
display. Here, assuming that the screen with the cursor display 
element 601 superimposed onto the screen display 500 for overlap 

25 display is the operating screen in a broad sense, it can be 



considered that the screen display data representing this broad 
operating screen (broad screen display data) is composed of two 
pieces of partial screen display data corresponding to the screen 
display 500 and the cursor display element 601. Furthermore, 
5 although the cursor display element takes a rectangular shape in 
the example shown in FIG. 20, it may take other shapes such as 
circle. Note that, in the present embodiment, bitmap data 
indicative of the cursor display element does not exist, and the 
shape, size, and color of the cursor display element is determined 
10 according to information included in the cursor display element 
information 81. 

The protocol of the network control system in the present 
embodiment is basically similar to that in the second embodiment 
as shown in FIG. 13. In the present embodiment, the screen 
15 information 111b sent from the device 200 to the controller 100 
as a response to the screen request 101 is structured as shown 
in FIG. 19 . The main screen information 65 and the cursor display 
element information 81 correspond to different screen display 
information in the second embodiment. After receiving such 
20 screen information, the controller 100 displays the operating 
screen on the display screen using the screen display data of the 
main screen information 65, and further superimposes the cursor 
display element onto the operating screen for overlap display 
using the information about the shape, size, and color, and the 
25 position information in the cursor display element information 
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81. 

After receiving the screen information 111b, when the 
operator carries out operation to the device through the operating 
screen as a user interface in the controller 100, the controller 

5 100 transmits to the device 200 the identification information 
of the screen display data indicative of the operating screen and 
the operation information indicative of the operation as the 
operation request 241. Alternatively, before transmitting the 
operation request 241, the controller 100 may check whether the 

10 user operation is allowed or not based on the operation enable 
information in the cursor display element information 81 received 
as the screen information 111b. As a result of this check, if 
the operation is not allowed, the controller 100 does not transmit 
the operation request 241 but carries out error handling, thereby 

15 preventing unnecessary processing. By way of example, if the 
cursor is located on contents 1 shown in FIG. 20, the operation 
by the leftward direction arrow key on the remote controller may 
be made invalid. Alternatively, if the user presses or releases 
a selection key on the remote controller, not "press" , "release" , 

20 but "select" may be sent to the device as the user operation 
information. Therefore, the processing of the device can be 
simplified. 

After issuance of the operation request 241, when the state 
in the device 200 is changed and therefore the screen display data 
25 in the device 200 is changed, the device 200 spontaneously 



transmits to the controller 100 the partial screen information 
121b corresponding to a changed part of the screen display data. 
In the present embodiment, if the cursor display element is 
changed by operation from the user, only the cursor display 
element information 81 as shown in FIG. 19 is sent to the controller 
100. After receiving, based on the position information and the 
information indicative of the shape, size, and color included in 
the cursor display element information 81, the controller 100 
superimposes the cursor display element onto the operating screen 
for overlap display. 

As described above, according to the present embodiment, 
when the cursor display element is changed by operation from the 
user, the device does not have to send bitmap data such as screen 
display data to the controller, thereby improving the response 
speed to the user operation. Furthermore, by setting information 
indicative of the shape, size, and color included in the cursor 
display element information 81, it is possible to conform the 
design of the cursor to the design of each display element on the 
operating screen. 
<Others> 

In each of the above described embodiments, units connected 
to the transmission path 1 are units handling at least one of video, 
audio, and information. However, units that do not directly 
relate to either video, audio, or information, such as air 
conditioner, may be connected to the transmission path 1 as 
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devices to be controlled. The present invention can be applied 
to a network control system with such units connected thereto. 

INDUSTRIAL APPLICABILITY 
5 The present invention is applied to a network control system 

in which a unit connected to a network is operated through the 
network. Specifically, the present invention is suitable for a 
device control system using a graphical user interface (GUI) that 
supports user's device operation through graphics, characters, 
10 etc. , on a screen. Such system includes a network control system 
in which a plurality of AV units are connected to each other a 
transmission path such as a serial bus of IEEE 1394 standard and 
one with a GUI controls the other units through the transmission 
path. 

15 
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WHAT IS CLAIMED IS: 



1 . A network control system in which a first unit and a second 
unit are connected to each other through a transmission path and 
a controller included in said first unit controls a device in said 
second unit through said transmission path, 
5 at least one of said first unit and said second unit being 

for handling at least one of video, audio, and information, 
said device 

having screen display data for displaying an operating 
screen of said device and identification information for 
10 identifying the screen display data; and 

transmitting said screen display data and said 
identification information to said controller through said 
transmission path, and 
said controller 

15 comprising a user interface including display means; 

receiving said screen display data and said 
identification information from said device through said 
transmission path; 

displaying said operating screen on said display means 
20 using said screen display data; and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
information indicative of the operation and said identification 
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information to said device through said transmission path. 

2 . A network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in said first unit controls a device in said 
second unit through said transmission path, 

at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, 

said device 

having screen display data for displaying an operating 
screen of said device and identification information for 
identifying the screen display data; and 

transmitting said screen display data and said 
identification information to said controller through said 
transmission path, and 
said controller 

comprising a user interface including display means; 

receiving said screen display data and said 
identification information from said device through said 
transmission path; 

displaying said operating screen on said display means 
using said screen display data and said identification 
information; and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
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information indicative of the operation and said identification 
information to said device through said transmission path. 

3. The network control system according to claim 1 or 2 , wherein 

said identification information includes version 
information indicating a version of said screen display data. 

4. The network control system according to claim 1 or 2, wherein 

said operation information includes an operating position 
information indicative of an operating position on said operating 
screen. 

5. A network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in said first unit controls a device in said 
second unit through said transmission path, 

at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, 
said device 

having screen display data composed of a plurality of 
partial screen display data for displaying an operating screen 
of said device; and 

transmitting said partial screen display data to said 
controller through said transmission path, and 
said controller 
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comprising a user interface including display means; 

receiving said partial screen display data from said 
device through said transmission path; 

displaying said operating screen on said display means 
using said partial screen display data; and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
information indicative of the operation to said device through 
said transmission path. 

6 . The network control system according to claim 5 , wherein 

when said screen display data of said device is changed, 
said device transmits changed partial screen display data of said 
screen display data to said controller; and 

said controller receives the changed partial screen display 
data from said device through said transmission path and, based 
on the received partial screen display data, updates said 
operating screen displayed on said display means. 

7. The network control system according to claim 5, wherein 

said device 

has partial screen identification information for 
identifying said partial screen display data; and 

transmits said partial screen display data and said 
partial screen identification information to said controller 
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through said transmission path, and 
said controller 

receives said partial screen display data and said partial 
screen identification information from said device through said 
transmission path. 

8 . The network control system according to claim 7 , wherein 

when said screen display data of said device is changed, 
said device transmits changed partial screen display data of said 
screen display data and the partial screen identification 
information of the partial screen display data to said controller, 
and 

said controller receives the changed partial screen display 
data and the partial screen identification information of the 
partial screen display data from said device through said 
transmission path and, based on the received partial screen 
display data and partial screen identification information, 
updates said operating screen displayed on said display means . 

9 - The network control system according to claim 7 , wherein 

in response to an operation by a user to said operating 
screen, said controller controls said device by transmitting 
operation information indicative of the operation and the partial 
screen identification information corresponding to the operation 
to said device through said transmission path. 
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10 . The network control system according to claim 7 or 9 , wherein 

said partial screen identification information includes 
version information indicating a version of said partial screen 
display data. 

11 . The network control system according to claim 7 or 9 , wherein 

one display element in screen display is arranged in any 
one of a plurality of display parts corresponding to said 
plurality of partial screen display data. 

12 . The network control system according to claim 7 or 9 , wherein 
a display element corresponding to each operation by the 
user is arranged in any one of a plurality of display parts 
corresponding to said plurality of partial screen display data. 

13. A network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in said first unit controls a device in said 
second unit through said transmission path, 
5 at least one of said first unit and said second unit being 

for handling at least one of video, audio, and information, 
said device 

having screen display data for displaying an operating 
screen of said device and overlap display data for overlap display 
10 on said operating screen; and 
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transmitting said screen display data and said overlap 
display data to said controller through said transmission path, 
and 

said controller 

15 comprising a user interface including display means; 

receiving said screen display data and said overlap 
display data from said device through said transmission path; 

displaying said operating screen on said display means 
using said screen display data, and carrying out overlap display 
20 on said operating screen displayed on said display means by using 
said overlap display data; and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
information indicative of the operation to said device through 
25 said transmission path. 

14. The network control system according to claim 13, wherein 

said overlap display data is cursor information for 
displaying a cursor indicative of a position of operation by the 
user to said operating screen. 

15. The network control system according to claim 14, wherein 

said cursor information includes position information 
indicative of a position of said cursor on said operating screen. 
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16. The network control system according to claim 14, wherein 

said cursor information includes shape information 
indicative of a shape of said cursor. 

17. The network control system according to claim 14, wherein 

said cursor information includes size information 
indicative of a size of said cursor. 

18. The network control system according to claim 14, wherein 

said cursor information includes color information 
indicative of a color of said cursor. 

19. The network control system according to claim 14, wherein 

said cursor information includes enable information 
indicating an operation that the user is allowed to perform. 

20. The network control system according to claim 13, wherein 

when a display part corresponding to said overlap display 
data of said device is changed, said device transmits said overlap 
display data to said controller, and 

said controller receives said overlap display data from 
said device through said transmission path and, based on said 
received overlap display data, updates said operating screen 
displayed on said display means . 
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21. A network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in said first unit controls a device in said 
second unit through said transmission path, 

at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, 

said device 

having screen display data for displaying an operating 
screen of said device and identification information for 
identifying the screen display data; and 

transmitting said screen display data and said 
identification information to said controller through said 
transmission path, and 
said controller 

comprising a user interface including display means; 

receiving said screen display data and said 
identification information from said device through said 
transmission path; and 

displaying said operating screen on said display means 
using said screen display data. 

22. A network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in said first unit controls a device in said 
second unit through said transmission path. 



5 at least one of said first unit and said second unit being 

for handling at least one of video, audio, and information, 
said device 

having screen display data for displaying an operating 
screen of said device and identification information for 
10 identifying the screen display data; 

transmitting said screen display data and said 
identification information to said controller through said 
transmission path, and 
said controller 

15 comprising a user interface including display means; 

receiving said screen display data and said 
identification information from said device through said 
transmission path; and 

displaying said operating screen on said display means 
20 using said screen display data and said identification 
information. 



23. A network control system in which a first unit and a second 
unit are connected to each other a transmission path and a 
controller included in said first unit controls a device in said 
second unit through said transmission path, 
5 at least one of said first unit and said second unit being 

for handling at least one of video, audio, and information, 
said device 
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having screen display data for displaying an operating 
screen of said device and overlap display data for overlap display 
on said operating screen; and 

transmitting said screen display data and said overlap 
display data to said controller through said transmission path, 
and 

said controller 

comprising a user interface including display means; 

receiving said screen display data and said overlap 
display data from said device through said transmission path; and 

displaying said operating screen on said display means 
using said screen display data, and carrying out overlap display 
on said operating screen displayed on said display means by using 
said overlap display data. 

24. A second unit that is connected through a transmission path 
to a first unit including a controller and includes a device 
controlled by said controller through said transmission path, 

at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, 

said device 

having screen display data for displaying an operating 
screen of said device and identification information for 
identifying the screen display data; 

transmitting said screen display data and said 
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identification information to said controller through said 
transmission path; and 

receiving the identification information of the screen 
display data and operation information indicative of an operation 
by a user, and operating based on the received identification 
information and operation information. 

25. A first unit that is connected through a transmission path 
to a second unit and includes a controller for controlling a device 
included in said second unit through said transmission path, 

at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, 
said controller 

comprising a user interface including display means; 

receiving screen display data indicative of an operating 
screen of said device and identification information for 
identifying the screen display data from said device through said 
transmission path; 

displaying said operating screen on said display means 
using said screen display data; and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
information indicative of the operation and said identification 
information to said device through said transmission path. 
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26. The second unit according to claim 24, wherein 

said operation information includes operating position 
information indicative of a position of operation on said 
operating screen. 

27. The first unit according to claim 25, wherein 

said operation information includes operating position 
information indicative of a position of operation on said 
operating screen. 

28. A second unit that is connected through a transmission path 
to a first unit including a controller and includes a device 
controlled by said controller through said transmission path, 

at least one of said first unit and said second unit being 
5 for handling at least one of video, audio, and information, 
said device 

having screen display data composed of a plurality of 
partial screen display data for displaying an operating screen 
of said device; 

10 transmitting said partial screen display data to said 

controller through said transmission path; and 

receiving operation information indicative of an 
operation by a user, and operating based on the received operation 
information . 
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29. A first unit that is connected through a transmission path 
to a second unit and includes a controller for controlling a device 
included in said second unit through said transmission path, 
at least one of said first unit and said second unit being 
5 for handling at least one of video, audio, and information, 
said controller 

comprising a user interface including display means; 
receiving a plurality of partial screen display data 
indicative of an operating screen of said device through said 
10 transmission path from said device; 

displaying said operating screen on said display means 
using said partial screen display data; and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
15 information indicative of the operation through said transmission 
path to said device. 

30. The second unit according to claim 28, wherein 

when said screen display data of said device is changed, 
said device transmits changed partial screen display data of said 
screen display data to said controller. 



31. The first unit according to claim 29, wherein 

when said screen display data of said device is changed, 
said controller receives changed partial screen display data of 
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said screen display data from said device through said 
5 transmission path and, based on the received partial screen 
display data, updates said operating screen displayed on said 
display means. 

32 . A second unit that is connected through a transmission path 
to a first unit including a controller and includes a device 
controlled by said controller through said transmission path, 

at least one of said first unit and said second unit being 
5 for handling at least one of video, audio, and information, and 
said device 

having screen display data for displaying an operating 
screen of said device and overlap display data for overlap display 
on said operating screen; 
10 transmitting said screen display data and said overlap 

display data to said controller through said transmission path; 
and 

receiving operation information indicating a user's 
operation transmitted from said controller, and operating based 
15 on the received operation information. 

33. A first unit that is connected through a transmission path 
to a second unit and includes a controller for controlling a device 
included in said second unit through said transmission path, 

at least one of said first unit and said second unit being 
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for handling at least one of video, audio, and information, 
said controller 

comprising a user interface including display means; 

receiving screen display data indicative of an operating 
screen of said device and overlap display data for overlap display 
on said operating screen through said transmission path; 

displaying said operating screen of said device on said 
display means using said screen display data, and performing 
overlap display on said operating screen displayed on said display 
means using said overlap display data; and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
information indicating the operation through said transmission 
path to said device. 

34 . A control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
path, at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, and 
a controller included in said first unit controlling a device 
included in said second unit through said transmission path, said 
control method comprising the steps of: 

transmitting screen display data for displaying an 
operating screen of said device and identification information 
for identifying the screen display data from said device through 
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said transmission path to said controller; 

displaying said operating screen on said controller using 
said screen display data transmitted from said device; and 

in response to an operation by a user to said operating 
15 screen, controlling said device by transmitting operation 
information indicative of the operation and said identification 
information from said controller through said transmission path 
to said device. 

35 . A control method, in a network control system in which a first 

unit and a second unit are connected to each other a transmission 

path, at least one of said first unit and said second unit being 

for handling at least one of video, audio, and information, and 
5 a controller included in said first unit controlling a device 

included in said second unit through said transmission path, said 

control method comprising the steps of: 

transmitting screen display data for displaying an 

operating screen of said device and identification information 
10 for identifying the screen display data from said device through 

said transmission path to said controller; 

displaying said operating screen on said controller using 

said screen display data and said identification information 

transmitted from said device; and 
15 in response to an operation by a user to said operating 

screen, controlling said device by transmitting operation 



information indicative of the operation and said identification 
information from said controller through said transmission path 
to said device. 

36 . A control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
path, at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, and 
5 a controller included in said first unit controlling a device 
included in said second unit through said transmission path, said 
control method comprising the steps of: 

transmitting at least one of plurality of partial screen 
display data composing screen display data for displaying an 
10 operating screen of said device from said device through said 
transmission path to said controller; 

displaying said operating screen on said controller using 
said partial screen display data transmitted from said device; 
and 

15 in response to an operation by a user to said operating 

screen, controlling said device by transmitting operation 
information indicative of the operation from said controller 
through said transmission path to said device. 

37 . A control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
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path, at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, and 
a controller included in said first unit controlling a device 
included in said second unit through said transmission path, said 
control method comprising the steps of : 

transmitting screen display data for displaying an 
operating screen of said device and overlap display data for 
overlap display on said operating screen from said device through 
said transmission path to said controller; 

displaying said operating screen on said controller and 
performing overlap display on said displayed operating screen 
based on said overlap display data by using said screen display 
data and said overlap display data transmitted from said device; 
and 

in response to an operation by a user to said operating 
screen, controlling said device by transmitting operation 
information indicative of the operation from said controller 
through said transmission path to said device. 

38 . A control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
path, at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, and 
a controller included in said first unit controlling a device 
included in said second unit through said transmission path, said 
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control method comprising the steps of: 

transmitting screen display data for displaying an 
operating screen of said device and identification information 
10 for identifying the screen display data from said device through 
said transmission path to said controller; and 

displaying said operating screen on said controller using 
said screen display data transmitted from said device. 

39. A control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
path, at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, and 
5 a controller included in said first unit controlling a device 
included in said second unit through said transmission path, said 
control method comprising the steps of : 

transmitting screen display data for displaying an 
operating screen of said device and identification information 
10 for identifying the screen display data from said device through 
said transmission path to said controller; and 

displaying said operating screen on said controller using 
said screen display data and said identification information 
transmitted from said device. 

40 . A control method, in a network control system in which a first 
unit and a second unit are connected to each other a transmission 
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path, at least one of said first unit and said second unit being 
for handling at least one of video, audio, and information, and 
a controller included in said first unit controlling a device 
included in said second unit through said transmission path, said 
control method comprising the steps of : 

transmitting screen display data for displaying an 
operating screen of said device and overlap display data for 
overlap display on said operating screen from said device through 
said transmission path to said controller; and 

displaying said operating screen on said controller and 
performing overlap display on said operating screen based on said 
overlap display data by using said screen display data and said 
overlap display data transmitted from said device. 
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ABSTRACT OF THE DISCLOSURE 

In a network control system, a device has screen display 
data indicative of its operating screen, and a controller issues 

5 a screen request (101) to obtain the screen display data, etc., 
as screen information (111) from the device. By using the screen 
display data, etc. , the controller displays the operating screen. 
When an operation is carried out to the operating screen, its 
operation information and identification information of the 

10 screen display data are sent to the device as operation request 
(241). The device executes a function corresponding to the 
operation request. As a result, when the screen display data in 
the device is changed, the device sends the changed screen display 
data, etc., to the controller as screen information (121). 
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